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EXECUTIVE  SUMMARY 


This  report  documents  the  results  of  the  first  Federal  Aviation 
Administration  (FAA)  manned  simulation  study  designed  to  demonstrate 
and  quantify  some  of  the  benefits  that  would  accrue  to  National  Airspace 
System  (NAS)  users  in  return  for  equipping  aircraft  to  receive  domestic, 
two-way  Data  Link  Air  Traffic  Control  (ATC)  services.  Air  traffic 
controllers,  ATC  supervisors,  and  professional  pilots  participated  in  high 
fidelity  simulation  tests  in  which  a  combined  Data  Link  and  voice  radio 
communication  system  was  used  to  control  traffic  in  en  route  airspace. 

The  study  demonstrated  that  controllers  using  two-way  Data  Link  were 
able  to  provide  ATC  services  that  improved  en  route  sector  productivity 
and  efficiency.  These  effects  were  reflected  in  reduced  aircraft  ground 
delay,  flight  time,  and  flight  distance  in  comparison  to  a  current 
operational  environment  using  only  voice  radio  communications.  In  all 
cases,  the  results  were  achieved  with  a  margin  of  safety  which  met,  or 
exceeded,  current  ATC  standards,  and  with  no  indication  of  excessive 
controller  workload.  Economic  cost  savings  associated  with  the  findings 
were  computed  to  determine  the  potential  magnitude  of  two-way  Data 
Link’s  impact  and  its  significance  to  NAS  users. 

APPROACH. 

The  study  addressed  two  operational  en  route  ATC  problems  which  havci 
been  attributed  to  limitations  of  the  communications  capability  of 
controllers  using  the  existing  voice  radio  system.  In  both  cases  it  was 
predicted  that,  by  expanding  the  capacity  of  the  communication  channel, 
Data  Link  would  permit  controllers  to  improve  the  efficiency  of  traffic  flow 
and  reduce  aircraft  delays. 

The  first  experiment  tested  the  ability  of  two-way  Data  Link  ATC 
communications  to  reduce  airport  departure  delays  that  are  caused  by 
capacity  problems  in  a  high  altitude  en  route  departure  sector.  It  was 
hypothesized  that  the  increased  communications  capability  provided  by 
Data  Link  would  alleviate  ground  delays  by  permitting  the  relaxation  of 
miles-in-trail  (MIT)  restrictions  which  are  routinely  implemented  for 
aircraft  entering  the  sector. 

The  second  experiment  tested  the  ability  of  the  Data  Link  to  improve  air 
traffic  throughput  in  an  en  route  sector  where  saturation  is  responsible  lor 
inefficient  processing  of  aircraft  arriving  at  a  major  airport.  Because  of 
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voice  frequency  congestion,  controllers  operating  this  sector  often  are 
unable  to  consistently  supply  aircraft  at  a  rate  which  meets  minimum 
restrictions  for  airport  arrivals.  In  this  case,  it  was  hypothesized  that 
effective  use  of  the  combined  voice  and  Data  Link  capability  would 
improve  the  timeliness  with  which  aircraft  are  passed  through  the  sector 
and  delivered  to  the  arrival  fix,  thereby  reducing  flight  costs  and  delays. 

The  two  experiments  employed  a  case  study  methodology  common  to 
proof-of-concept  research.  Baseline  Data  Link  test  scenarios  were  built  to 
precisely  duplicate  air  traffic  sample  days  taken  from  two  airspace  sectors 
within  the  Atlanta  Air  Route  Traffic  Control  Center  (ARTCC)  which 
currently  experience  these  problems.  In  both  the  departure  and  arrival 
cases,  data  on  the  delays  experienced  by  traffic  on  the  sample  day  were 
derived  from  the  historical  System  Analysis  Recording  (SAR)  tapes. 
Additional  test  runs  were  conducted  using  increased  traffic  levels 
representative  of  future  demand. 

Staffing  for  the  two  sectors  was  identical  to  that  used  for  the  peak  traffic 
periods  recorded  on  the  sample  day  SAR  tapes  from  the  Atlanta  ARTCC. 
Teams  of  three  Atlanta  controllers  participated  in  the  radar,  data,  and 
tracker  control  positions.  Each  team  was  accompanied  by  an  ATC 
supervisor.  In  order  to  insure  that  the  results  were  repeatable,  three 
replications  of  the  experiment  were  completed  with  different  controller 
teams  over  the  6-week  study  period. 

KEY  RESULTS  -  EXPERIMENT  1. 

Experiment  1  addressed  a  sector  in  which  the  primary  task  is  to  control 
departures  of  jet  aircraft  climbing  from  the  Atlanta  airport.  During  the 
daily  peak  traffic  period  addressed  in  this  study,  traffic  saturation  currently 
requires  the  issuance  of  a  20  MIT  restriction  for  departures  entering  the 
sector.  This  situation  creates  significant  ground  delays  for  aU  aircraft 
preparing  to  take  off  from  the  runway  feeding  the  sector. 

During  testing  with  the  combined  Data  Link  and  voice  radio  system,  the 
subject  controllers  attempted  to  control  comparable  traffic  at  the  current 
20  MIT  restriction,  and  with  the  traffic  arriving  at  the  sector  boundaries  at 
times  corresponding  to  15  MIT,  10  MIT,  and  with  no  restriction  in  force.  In 
a  fifth  test  run,  Atlanta  departures  were  increased  by  10  percent  with  no 
restriction  in  force. 
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The  results  showed  that  the  control  teams  in  all  three  replications  of  the 
experiment  were  able  to  safely  and  effectively  control  the  air  traffic  at  each 
reduced  spacing  restriction,  and  with  a  10-percent  increase  in  departure 
traffic.  No  aircraft  separation  violations  were  detected  by  the  Host 
computer  under  any  of  the  test  conditions.  Furthermore,  none  of  the 
operational  assessments  performed  by  the  supervisors,  controllers,  or 
simulator  pilots  revealed  any  indication  that  the  observed  improvements  in 
traffic  handling  capability  were  achieved  with  any  compromise  in  the 
safety  of  ATC  operations.  A  majority  of  the  operational  assessment  ratings 
performed  by  the  supervisors  suggested  that  the  margin  of  safety  was 
greater  than  normal  for  sector  during  the  peak  traffic  period  when  Data 
Link  was  used. 

The  direct  user  benefit  associated  with  the  performance  achieved  by  the 
control  teams  using  Data  Link  when  no  restrictions  were  placed  on 
departing  aircraft  was  a  dramatic  reduction  in  ground  delays.  Total  delays 
for  all  departing  aircraft  decreased  from  1,795  minutes  with  the  20  MIT 
restriction  currently  imposed  at  Atlanta  to  687  minutes  when  the  use  of 
Data  Link  permitted  the  elimination  of  restrictions.  Comparable  savings 
were  obtained  with  a  10-percent  increase  in  traffic. 

Measures  of  in-flight  traffic  flow  indicated  that  the  ground  delay  savings 
were  not  sacrificed  by  inefficient  control  of  the  aircraft  once  they  had 
entered  the  sector.  Moreover,  the  data  show  that  using  Data  Link,  the  test 
controllers  were  able  to  provide  service  which  reduced  flight  time  and 
distance  by  20  percent  for  all  aircraft  in  the  sector  compared  to  their 
counterparts  using  only  voice  communications  on  the  representative 
historical  baseline  day. 

KEY  RESULTS  -  EXPERIMENT  2. 

In  experiment  2,  Data  Link  testing  was  conducted  in  an  arrival  sector 
where  the  primary  task  is  to  sequence  aircraft  inbound  to  the  Atlanta 
airport.  During  two  daily  peak  periods,  traffic  saturation  typically  creates 
a  situation  in  which  controllers  cannot  consistently  provide  minimum  MIT 
to  aircraft  exiting  the  sector  into  approach  control  airspace.  The  volume  of 
required  communications  often  prevents  efficient  merging  of  traffic  and 
forces  the  use  of  holding  patterns  and  extensive  vectoring.  This  delays 
arrivals  and  results  in  excess  fuel  consumption. 

During  initial  testing  with  a  combined  Data  Link  and  voice  radio 
communications  system,  Atlanta  controllers  attempted  to  control  traffic  in 
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a  scenario  which  reproduced  the  air  traffic  demand  recorded  for  sector  09 
on  the  historical  SAR  tape  for  one  of  the  daily  peak  periods.  Subsequent 
test  trials  increased  the  volume  of  arrivals  by  from  10  to  40  percent. 

The  results  indicate  that  all  three  teams  of  test  controllers  were  able  to 
control  the  arriving  Atlanta  aircraft  in  sector  09  more  efficiently  with  Data 
Link  than  their  counterparts  using  only  voice  radio  on  the  historical 
baseline  day.  While  the  average  arrival  on  the  baseline  day  flew 
approximately  111  miles  over  18  minutes  in  the  sector,  the  average 
controller  team  using  Data  Link  was  able  to  reduce  the  distance  flown  to 
89  miles  over  14  minutes  with  the  identical  traffic  sample.  When  the  traffic 
sample  was  increased  by  up  to  40  percent,  the  data  show  that  the  arrival 
aircraft  maintained  an  advantage  of  3  to  4  fewer  minutes  of  flight  time  in 
the  sector  when  Data  Link  was  used  to  expand  the  ATC  communications 
channel.  In  all  cases,  these  improvements  were  achieved  through  more 
strategic  sequencing  of  the  arrival  traffic  streams,  the  elimination  of 
holding  patterns,  and  more  direct  clearances  to  the  arrival  fix. 

Control  teams  in  all  three  replications  of  the  experiment  controlled  the  air 
traffic  safely  and  effectively.  No  aircraft  separation  violations  were 
recorded.  Furthermore,  their  were  no  indications  that  the  observed 
improvements  in  sector  throughput  were  achieved  with  any  reduction  in 
operational  safety.  All  supervisor,  controller,  and  pilot  operational 
assessments  indicated  that  the  margin  of  safety  during  the  Data  Link  test 
was  equal  to,  or  greater  than,  that  experienced  at  the  actual  sector  during 
the  peak  arrival  traffic  period  using  only  voice  communications. 

ADDITIONAL  FINDINGS. 

A  variety  of  subsidiary  measures  were  collected  during  both  experiments 
to  support  the  primary  benefit  measures.  ATC  supervisor  evaluations  of 
key  sector  tasks  indicated  that  controller  performance  was  within  normal 
limits,  or  better,  for  all  Data  Link  test  runs.  Furthermore,  controller 
ratings  and  measures  of  handoff  latency  yielded  no  evidence  of  excessive 
controller  workload  with  Data  Link. 

The  performance  benefits  demonstrated  in  the  experiments  were 
associated  with  reductions  of  78  to  84  percent  in  voice  radio  channel 
occupation  time  as  Data  Link  was  used  to  send  most  clearances  and 
transfer  of  communication  messages  to  equipped  aircraft. 

In  addition,  the  data  show  that  the  test  control  teams  distributed 
communications  responsibilities  among  all  three  members,  permitting 
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optimal  use  of  the  expanded  communications  channel  and  increased  sector 
productivity. 

NAS  USER  COST  SAVINGS. 

The  direct  effect  of  Data  Link  addressed  in  this  study  is  its  ability  to 
increase  the  capacity  of  the  communications  channel  between  controllers 
and  pilots.  When  applied  to  en  route  airspace  sectors  currently  affected  by 
frequency  congestion,  the  results  of  the  study  showed  that  by  permitting 
more  timely  and  effective  clearance  delivery,  a  communications  system 
combining  voice  radio  and  Data  Link  can  reduce  aircraft  delays  and 
increase  operating  efficiency. 

The  final  task  of  this  study  was  to  translate  the  system  performance 
improvements  observed  in  the  experiments  to  estimates  of  the  annual  cost 
savings  that  would  be  received  by  NAS  users.  Initial  estimates  were 
computed  directly  from  the  performance  data  for  the  test  sectors  at  the 
Atlanta  ARTCC.  In  both  cases,  these  projections  were  restricted  to  the  two 
sectors  over  the  two,  1-hour  peak  traffic  periods.  The  predictions  were 
based  on  current  FAA  statistical  computations  of  the  average,  per-minute 
cost  to  air  carriers  for  taxi  and  in-flight  delays.  Total  savings  attributable  to 
the  reduced  ground  delays  and  flight  times  demonstrated  with  Data  Link 
for  the  two  sectors  were  in  excess  of  8.6  million  dollars  annually. 

An  estimate  of  national  annual  benefit  was  computed  from  the  results  of  a 
national  sector  survey  and  the  findings  of  the  two  experiments  conducted 
for  this  study.  The  results  show  that  over  9,600  flights  are  affected  nearly 
every  day  by  communications-related  saturation  in  the  43  sectors  identified 
by  the  survey.  These  flights  experience  7,980  aircraft-days  of  delay  per 
year.  The  total  estimated  annual  cost  savings  to  NAS  users  associated 
with  implementing  two-way  Data  Link  ATC  communications  would  be  over 
337  million  dollars. 

In  evaluating  the  significance  of  these  findings,  it  should  be  remembered 
that  the  calculated  cost  savings  are  restricted  to  Data  Link’s  ability  to 
increase  ATC  communications  capacity  in  a  single  domain  of  ATC 
operations.  Safety  and  efficiency  enhancements  that  may  result  from  two- 
way  Data  Link’s  capability  to  improve  the  accuracy  of  communications, 
and  from  its  use  in  other  ATC  environments  are  expected  to  significantly 
increase  estimated  cost  savings  to  aircraft  owners  and  operators. 
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1.  INTRODUCTION. 


LI  PURPOSE. 

This  report  documents  the  results  of  the  first  in  a  planned  series  of  studies 
designed  to  assess  benefits  associated  with  the  implementation  of 
domestic,  two-way  Data  Link  air  traffic  control  (ATC)  services.  Real-time, 
manned  simulation  testing  was  used  to  measure  some  of  the  quantifiable 
benefits  that  would  accrue  to  aircraft  owners  and  operators  for  investing  in 
avionics  equipment  to  receive  these  services.  Incidence  data  collected 
from  ATC  facilities  throughout  the  National  Airspace  System  (NAS)  were 
then  employed  to  produce  nationalized  estimates  of  net  annual  benefits 
associated  with  the  test  results. 

The  work  reported  here  specifically  focused  on  the  effects  of  introducing 
Data  Link  communications  to  en  route  ATC  environments  where  aircraft 
delay  and  operating  efficiency  are  influenced  by  saturation  of  the  air- 
ground  communications  channel.  Future  studies  and  analyses  will 
exEimine  additional  benefits  and  other  ATC  environments. 

1.2  BACKGROUND. 

1.2.1  Data  Link  Implementation  Plans. 

Over  the  past  several  years,  the  Federal  Aviation  Administration  (FAA)  has 
evolved  a  comprehensive  plan  for  building  an  air  traffic  management 
(ATM)  system.  This  system  will  support  future  global  flight  planning, 
aircraft  operation,  and  ATC  services  through  the  introduction  of  advanced 
communications,  navigation,  and  surveillance  technologies.  A  key  feature 
of  the  future  ATM  system  will  be  the  use  of  digital  data  communications  as 
a  primary  means  for  exchanging  aeronautical  information  and  delivering 
ATC  services. 

The  FAA  plan  currently  calls  for  implementation  of  domestic,  in-flight  Data 
Link  ATC  services  to  begin  in  1998  with  the  introduction  of  two-way 
communications  in  the  en  route  and  terminal  ATC  environments.  At  this 
initial  stage,  controllers  will  have  the  capability  to  uplink  a  variety  of 
clearance  and  advisory  messages  to  equipped  aircraft,  and  aircrews  will  be 
able  to  downlink  reports  and  ATC  requests.  Performance  standards  and  a 
message  dictionary  for  these  two-way  Data  Link  communications  have 
been  published  by  RTCA  (1993). 
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Within  the  next  several  years,  Data  Link  implementation  will  begin  to 
expand  to  a  broad  range  of  automated  information  exchanges.  These  end- 
state  applications  will  provide  a  connection  between  airborne  and  ground- 
based  data  and  computing  resources  that  will  optimize  approach  traffic 
flow,  enable  the  routine  use  of  4-D  navigation  and  permit  long  range 
conflict  detection  and  resolution. 

Execution  of  the  plan  outlined  above  will  require  extensive  cooperation 
between  government  and  private  industry,  and  major  resource  investments 
by  both  parties.  The  FAA  must  create  the  integrated  telecommunications 
network  needed  to  support  Data  Link  and  develop  associated  ATC 
software  and  controller  interfaces.  In  order  to  justify  the  ground-based 
investment,  users  of  the  NAS  must  make  a  simultaneous  decision  to  equip 
a  significant  number  of  aircraft  with  the  avionics  and  aircrew  interfaces 
needed  for  Data  Link  communications. 

1 .2 .2  Justification  for  Early  Data  Link  Equipage. 

The  success  of  the  phased  introduction  of  Data  Link  ATC  services 
promoted  by  the  FAA  plan  will  depend  on  growth  in  airborne  equipage 
levels  during  the  initial  stage.  This  early  build-up  will  permit  thorough 
validation  of  the  communications  system,  making  the  later  introduction  of 
Data  Link-mediated  automation  more  technically  feasible  and  cost- 
effective. 

There  is  general  agreement  among  both  government  and  NAS  user 
representatives  that  significant  improvements  in  the  efficiency  of  airborne 
operations  will  be  achieved  when  Data  Link  is  used  to  enable  advanced 
automation.  However,  because  of  the  current  lack  of  evidence  for 
immediate  economic  benefits  associated  with  domestic  two-way  Data  Link 
ATC  services,  the  case  for  equipping  aircraft  during  the  first  stage  of 
implementation  has  not  met  with  widespread  acceptance.  In  response  to 
these  concerns,  the  FAA  has  elected  to  initiate  a  program  of  a  real-time, 
manned  simulation  research  to  identify  and  quantify  benefits  that  will 
accompany  the  implementation  of  a  two-way  Data  Link  system  for 
controller-pilot  ATC  communications.  The  study  described  in  this  report 
was  conducted  by  the  FAA  to  address  these  overall  goals. 

1 .3  DIRECT  EFFECTS  OF  DATA  LINK. 

Any  economic  benefits  that  may  be  associated  with  the  initial  two-way 
Data  Link  capability  will  be  realized  indirectly  through  changes  in  ATC 
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capabilities  and  performance  that  are  made  possible  by  this  technology. 

For  this  reason,  the  process  of  defining  a  study  to  assess  these  benefits 
began  by  examining  some  problems  which  exist  in  ATC  communications, 
and  the  effects  that  the  introduction  of  Data  Link  can  be  expected  to  have 
on  them. 

1.3.1  Frequency  Congestion. 

One  of  the  primary  factors  that  has  driven  the  development  of  Data  Link 
has  been  the  growing  utilization  of  voice  radio  in  the  NAS.  Because  the 
voice  channel  is  available  to  only  one  speaker  at  a  time,  increasing  traffic 
volume  rapidly  leads  to  frequency  congestion.  When  the  channel  becomes 
saturated,  system  performance  suffers  as  ATC  clearances  become  less 
timely  and  the  exchange  of  information  is  reduced  to  an  absolute 
minimum. 

In  order  to  understand  how  Data  Link  is  likely  to  affect  ATC  system 
capabilities,  it  is  important  to  examine  the  way  in  which  controller 
performance  is  affected  by  frequency  congestion.  When  debriefing 
controllers  about  the  factors  which  influence  workload  experienced  on  the 
job,  the  demands  associated  with  monitoring  the  radar  display  or  making 
decisions  about  appropriate  control  actions  are  rarely  mentioned.  Instead, 
it  is  common  to  hear  that  "it's  not  the  number  of  aircraft  in  my  sector  that 
creates  workload,  but  what  I  have  to  do  with  them  and  how  much  time  I 
have  to  do  it." 

Such  statements  emphasize  a  close  relationship  between  workload  and  the 
amount  of  communication  required  to  maintain  control  over  air  traffic. 
They  also  suggest  that  any  factor  which  limits  the  amount  of  time  available 
to  perform  required  communications  with  aircraft  will  increase  controller 
workload  or  reduce  the  efficiency  with  which  ATC  tasks  can  be 
accomplished.  As  frequency  usage  is  increased,  high  workload  is 
precipitated  by  a  proportional  limitation  in  the  time  available  to  carry  out 
required  communications,  rather  than  an  inherent  inability  of  the 
controller  to  handle  the  air  traffic  or  by  saturation  of  the  airspace  itself. 

From  a  theoretical  point  of  view,  this  workload  problem  can  be  seen  as  a 
special  combination  of  the  data  limitations  and  resource  limitations  on 
human  performance  discussed  by  Norman  and  Bobrow  (1975).  In  the  case 
of  frequency  congestion,  constraints  imposed  by  the  simplex  nature  of  the 
radio  system  produce  a  data  limitation  on  controller  performance  by 
preventing  timely  communications.  That  is,  performance  is  limited  by  the 
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physical  constraints  of  the  system  rather  than  the  information  processing 
resources  of  the  controller.  However,  as  a  result  of  the  shrinking 
performance  windows,  mental  resource  limitations  come  into  play  and 
workload  increases  as  the  controller  attempts  to  maintain  system 
performance  by  emplojdng  tactical  control  strategies  which  tax  his 
perceptual,  cognitive,  and  speech-motor  capacities. 

Because  of  the  close  coupling  between  the  controller's  ability  to  make 
effective  use  of  available  system  capacity  and  voice  frequency  usage,  a 
fundamental  measurement  taken  during  high  fidelity  Data  Link 
simulations  at  the  FAA  Technical  Center  has  been  the  extent  to  which  Data 
Link  reduces  use  of  the  radio  channel  to  communicate  with  aircraft.  In 
each  of  these  studies,  subject  controllers  are  asked  to  control  simulated  air 
traffic  using  traditional  voice  communications  alone,  and  under  conditions 
where  both  voice  and  Data  Link  channels  are  available.  During  Data  Link 
trials,  the  controllers  are  instructed  to  perform  the  ATC  task  as  safely  and 
efficiently  as  possible,  and  to  use  the  communications  system  that  they  feel 
is  most  effective  for  accomplishing  their  objectives. 

Data  Link's  effect  on  voice  radio  frequency  usage  has  been  measured  in 
both  en  route  and  terminal  simulation  studies.  In  an  en  route  operational 
evaluation,  the  provision  of  an  initial  service  capability  (transfer  of 
communication  and  altitude  assignment)  reduced  the  number  of  voice 
transmissions  initiated  by  controllers  up  to  41  percent.  It  also  reduced  the 
total  amount  of  time  that  the  controllers  occupied  the  radio  frequencies  up 
to  45  percent  (Talotta,  et.  al.,  1990).  Furthermore,  as  the  proportion  of 
aircraft  in  the  test  scenarios  equipped  with  Data  Link  was  raised  from  20  to 
80  percent  of  the  total  number  presented  in  the  test  scenario,  the  overall 
efficiency  of  ATC  communications  improved  as  requirements  for 
repetitions  of  voice  messages  and  clarification  of  misunderstood  clearances 
decreased. 

Similar  dramatic  reductions  in  voice  radio  usage  were  obtained  in  terminal 
testing  under  75  percent  aircraft  Data  Link  equipage.  In  this  situation, 
where  controllers  could  transmit  speeds,  headings,  and  initial  contact 
responses  as  well  as  the  services  used  in  the  en  route  study,  the  number  of 
voice  messages  issued  by  controllers  dropped  by  50  percent  over  the  voice- 
only  test  conditions,  and  radio  channel  occupation  time  by  controllers  fell 
by  60  percent  (Data  Link  Development  Team,  1991). 

Taken  together,  the  results  of  the  studies  cited  above,  and  others  obtained 
in  high  fidelity  simulation  research  at  the  FAA  Technical  Center,  make  two 
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important  points  about  Data  Link  ATC  communications.  First,  they 
demonstrate  that,  given  an  appropriate  human-computer  interface,  ATC 
communications  using  Data  Link  can  be  conducted  effectively  in  the 
context  of  the  air  traffic  controller’s  complex  and  dynamic  task 
environment.  Second,  the  findings  show  that  controllers  can  utilize  the 
traditional  voice  channel  along  with  Data  Link  to  effectively  expand  the 
capacity  of  the  air-ground  communications  channel. 

1.3.2  Communications  Errors. 

In  addition  to  its  potential  for  reducing  communications-induced 
limitations  on  effective  system  capacity,  analyses  performed  by  the  FAA 
Technical  Center  also  have  indicated  that  Data  Link  has  the  ability  to 
reduce  the  occurrence  of  common  ATC  communications  errors  that  affect 
flight  safety  and  efficiency.  Incidence  estimates  available  from  a  number 
of  sources  clearly  show  that  communications  problems  are  a  major  source 
of  concern  in  the  present  ATC  system.  In  1988,  the  FAA  noted  that  23 
percent  of  all  operational  errors  (minimum  aircraft  separation  violations) 
were  caused  either  directly  or  indirectly  by  communications  mistakes. 
Similarly,  compilations  of  reports  provided  on  a  voluntary  basis  by  aircrew 
and  controllers  to  the  Aviation  Safety  Reporting  System  (ASRS)  have 
indicated  that  70  to  80  percent  of  all  potentially  hazardous  incidents  that 
are  reported  implicate  ineffective  verbal  information  transfer;  and  that  a 
clear  majority  of  these  involve  air-ground  radio  communications  (Billings 
and  Reynard,  1981;  FAA,  1988). 

Common  categories  of  human  error  which  appear  to  be  the  primary 
sources  of  the  cited  communications  problems  include  acoustic  confusion 
and  transposition  of  alphanumerics,  pilot  readback  error,  controller 
"hearback"  error,  misinterpretation  caused  by  poor  pronunciation  and 
failure  to  use  standard  phraseology,  and  improper  radio  keying  technique. 

Shingledecker  and  Talotta,  (1993)  discussed  how  several  general  human 
performance  limitations  appear  to  combine  with  the  features  of  a  simplex 
voice  radio  system  to  promote  the  errors  that  are  commonly  observed  in 
the  ATC  environment.  These  traits  include:  (1)  the  limited  rate  at  which 
humans  can  produce  and  comprehend  the  speech  signal  (partially  defining 
the  point  at  which  radio  frequency  congestion  becomes  problematic),  (2) 
short  term  memory  limits  for  the  content  of  the  transient  acoustic  displays 
used  in  radio  communications,  (3)  the  tendency  to  rely  on  expectation  in 
the  absence  of  unambiguous  data,  (4)  human  susceptibility  to  phonetic 
confusion  in  acoustic  displays,  (5)  the  relative  unreliability  of  the  human  as 
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a  monitoring  device  in  a  multiple  task  environment,  and  (6)  the  tendency 
for  humans  in  a  high  workload  situation  to  adapt  by  shedding  load,  often 
sacrificing  highly  redundant,  prescribed  communications  formats. 

Unlike  voice  radio.  Data  Link  offers  a  communications  medium  which 
transmits  coded,  digitEd  data  to  individual  addressees.  This  feature  of  Data 
Link  can  be  expected  to  alleviate  the  problems  induced  by  the  human- 
system  interaction  at  nearly  all  stages  of  the  communications  process. 
While  Data  Link  cannot  compensate  for  poor  controller  decisions,  the 
message  formulation  stage  should  be  improved  by  providing 
reasonableness  and  logic  checks  of  the  digital  data.  Message  composition 
can  be  assisted  by  storing  common  messages  for  selection  from  a  menu, 
and  by  employing  automatic  checks  on  controller  input  formats  to  prevent 
the  transmission  of  ambiguous  clearances.  Furthermore,  message 
composition  would  not  be  impeded  by  the  delays  experienced  when  the 
radio  channel  is  in  use  by  pilots. 

Message  transmission  also  will  be  improved  because  Data  Link  will 
assume  some  portion  of  the  load  on  congested  radio  frequencies.  As 
discussed  in  the  previous  section  of  this  paper,  this  will  not  only  increase 
the  availability  of  the  voice  frequency,  but  also  reduce  controller  workload 
and  increase  the  timeliness  of  clearance  delivery  by  permitting  controllers 
to  communicate  when  necessary  -  not  merely  when  the  channel  is 
available.  In  addition,  those  messages  carried  by  Data  Link  will  be 
effectively  immune  to  degradation  by  noise  and  blocking  that  plague  an 
analogue  radio  system  and  impair  pilot  perception.  Monitoring  failures 
and  message  (call  sign)  detection  by  the  pilot  will  be  totally  eliminated  as  a 
source  of  error  since  this  task  will  be  assumed  by  Data  Link's  discrete 
addressing  system.  Likewise,  message  interpretation  will  be  enhanced 
because  pilots  will  have  a  persistent,  storable  reference  of  message 
content,  and  because  available  evidence  suggests  that  a  visual  display  may 
be  less  prone  to  misinterpretation  than  an  acoustic  display. 

Finally,  the  acknowledgment  and  verification  stage  of  the  communications 
process,  which  is  a  human  responsibility  in  the  voice  radio  system,  will  be 
largely  allocated  to  Data  Link.  However,  rather  than  being  assumed  by 
Data  Link  in  an  analogous  sense,  the  verification  process  will  be  built  in 
each  transmission  as  the  system  automatically  verifies  the  integrity  of 
message  content  reaching  the  pilot,  assures  the  controller  that  the 
responding  aircraft  is  the  intended  receiver,  and  monitors  for  failed 
confirmatory  downlinks. 
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Based  on  the  analysis  summarized  above,  Shingledecker  and  Talotta 
(1993)  attempted  to  estimate  the  extent  to  which  the  introduction  of  Data 
Link  will  ameliorate  communications  errors.  Using  ASRS  data  collected 
between  1980-1984  including  2,700  reports  of  communications  problems, 
their  findings  indicate  that  Data  Link  would  produce  a  major  reduction  in 
communications  problems  that  form  45  percent  of  all  reported 
communications  incidents.  These  include  ambiguous,  incomplete,  and 
garbled  messages,  failures  to  detect  clearances,  phonetic  confusions,  and 
transposition  errors.  A  further  54  percent  of  incidents  would  be  at  least 
partially  reduced  by  Data  Link.  These  include  untimely  issuance  of 
clearances  caused  by  congested  or  blocked  frequencies  and  those  cases 
where  aircraft  perform  uncleared  maneuvers  because  of  confusions  about 
the  intended  receiver  of  a  message  or  because  of  misinterpretation  of  the 
clearance  itself.  Only  1  percent  of  all  problems  would  be  unaffected,  these 
being  situations  where  the  controller  issues  a  logically  reasonable,  but 
erroneous  clearance  because  of  faulty  decision  making. 

1.4  STUDY  RATIONALE. 

The  data  and  analyses  discussed  above  suggest  that  the  introduction  of 
two-way  Data  Link  ATC  services  will  significantly  improve  NAS  operations 
by  reducing  communications  errors  and  by  increasing  communications 
channel  capacity.  However,  the  task  of  using  these  indications  of  the 
direct  effects  of  Data  Link  to  produce  estimates  of  their  associated 
economic  benefits  is  complicated  by  the  fact  that  the  benefits  are  clearly 
context  specific.  Unlike  the  addition  of  a  new  parallel  runway  at  an  airport 
or  the  development  of  a  more  fuel-efficient  aircraft  engine,  two-way  Data 
Link  ATC  communications  are  not  expected  to  save  money  on  every  flight 
or  to  prevent  all  airborne  mishaps.  Instead,  the  observable  benefits  of  Data 
Link  will  be  displayed  only  under  those  conditions  where  restrictions  in 
communications  capability  and  communications  errors  significantly  affect 
the  quality  and  cost  of  aircraft  operation. 

Because  of  this,  the  study  described  in  this  report  was  based  on  the 
assumption  that  sensitive  measurements  of  the  economic  impact  of  Data 
Link  would  not  be  obtained  by  randomly  sampling  ATC  scenarios 
occurring  in  the  NAS.  Rather,  these  measures  would  be  required  to  focus 
on  those  specific  ATC  situations  and  conditions  under  which  Data  Link’s 
direct  effects  would  be  expected  to  ameliorate  the  underlying  cause  of 
inefficiency  or  delay.  Thus,  the  approach  developed  for  this  study  involved 
testing  under  a  sample  of  ATC  scenarios  in  which  Data  Link's  known 
effects  are  expected  to  translate  directly  to  quantifiable  economic  benefits. 
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To  insure  that  these  scenarios  were  representative  of  existing  air  traffic 
phenomena,  they  were  selected  from  recent  historical  records  of  actual 
NAS  operations. 

2.  STUDY  OBJECTIVES. 

The  primary  objective  of  this  study  was  to  measure  and  demonstrate  some 
of  the  benefits  that  will  be  gained  by  commercial  NAS  users  in  return  for 
equipping  aircraft  to  participate  in  two-way  Data  Link  ATC 
communications.  Specifically,  the  study  employed  real-time,  high  fidelity 
manned  ATC  simulation  to  assess  the  extent  to  which  the  increased 
communications  capacity  provided  by  Data  Link  can  (1)  reduce  aircraft 
spacing  restrictions  and  departure  delays,  and  (2)  reduce  aircraft  operating 
inefficiencies  when  used  in  two  en  route  airspace  sectors  where  ATC 
operations  are  routinely  limited  by  communications-related  sector 
saturation. 

A  second  objective  of  this  study  was  to  use  the  local  benefits  obtained  from 
the  empirical  results  of  the  experiments  to  project  system-wide  benefits 
under  current  and  future  air  traffic  demand  scenarios. 

Finally,  the  study  was  used  as  an  opportunity  to  address  a  number  of 
subsidiary  objectives  associated  with  the  use  of  Data  Link  at  en  route 
sectors  manned  by  multiple  air  traffic  controllers.  These  included  an  initial 
exploration  of  communications  procedures  for  control  teams,  and  an 
identification  of  human-computer  interface  requirements  to  support 
optimal  team  performance. 

3.  SUMMARY  OF  APPROACH. 

In  the  study  described  in  the  succeeding  sections  of  this  paper,  two 
operational  ATC  problems  were  addressed.  These  problems  have  been 
attributed  to  limitations  of  the  communications  capability  of  controllers 
using  the  existing  voice  radio  system.  In  both  cases,  the  benefits  of  Data 
Link  were  hypothesized  to  be  associated  with  decreases  in  aircraft  delays 
and  flight  time  made  possible  by  the  expanded  communications  channel. 
Anticipated  benefits  resulting  from  the  enhancement  of  communications 
accuracy  and  related  safety  improvements  were  not  explicitly  considered 
in  this  study. 

The  first  experiment  tested  the  ability  of  two-way  Data  Link  ATC 
communications  to  reduce  airport  delays  that  are  caused  by  capacity 
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problems  in  a  high  altitude  en  route  departure  sector.  It  was  hypothesized 
that  the  increased  communications  capability  provided  by  Data  Link  would 
alleviate  ground  departure  delays  by  permitting  the  relaxation  of  spacing 
restrictions  which  are  routinely  implemented  on  traffic  entering  the  subject 
sector. 

The  second  experiment  tested  the  ability  of  Data  Link  to  improve  air  traffic 
throughput  in  an  en  route  sector  where  sector  saturation  is  responsible  for 
inefficient  processing  of  aircraft  arriving  at  a  major  airport.  Because  of 
voice  frequency  congestion,  controllers  operating  these  sectors  often  are 
unable  to  consistently  supply  aircraft  at  a  rate  which  meets  minimum 
restrictions  for  airport  arrivals.  In  this  case,  it  was  h5q)othesized  that 
effective  use  of  the  combined  voice  and  Data  Link  capability  would 
improve  the  timeliness  with  which  aircraft  are  delivered  to  the  arrival  fix, 
thereby  reducing  flight  costs  and  Eirrival  delays. 

The  two  experiments  employed  a  case  study  methodology  common  to 
proof-of-concept  research.  Rather  them  synthesizing  scenarios  based  on  a 
general  concept  of  the  types  of  ATC  problems  discussed  above,  test 
scenarios  were  built  to  precisely  duplicate  air  traffic  data  sample  days 
taken  from  two  airspace  sectors  within  the  Atlanta  Air  Route  Traffic 
Control  Center  (ARTCC  )  which  currently  experience  these  problems.  For 
the  departure  case  (experiment  1),  baseline  data  on  the  delays  experienced 
by  traffic  on  the  sample  days  were  derived  from  the  historical  System 
Analysis  Recording  (SAR)  tapes.  The  tapes  also  were  used  to  create  Data 
Link  test  scenarios  which  present  departure  demands  identical  to  those 
experienced  on  the  sample  day.  During  testing,  miles -in-trail  (MIT) 
restrictions  were  incrementally  reduced  to  determine  the  capability  of 
subject  controllers  to  handle  the  increased  flow  using  the  combined  voice 
and  Data  Link  communications  systems.  The  difference  between  aircraft 
ground  delays  experienced  in  the  historical  baseline  when  the  standard 
MIT  restriction  was  introduced  and  those  achieved  in  simulation  with  Data 
Link  were  used  to  determine  accrued  benefits. 

For  the  arrival  problem  (experiment  2),  baseline  data  on  aircraft  flight 
paths  and  arrival  delays  for  the  affected  sector  were  computed  from 
information  contained  in  the  historical  SAR  tapes  for  the  sample  day.  As 
in  the  departure  problem,  the  SAR  tapes  also  were  used  to  create  Data 
Link  test  scenarios  that  mimicked  the  original  aircraft  crossings  into  the 
sector.  During  testing,  data  were  collected  to  assess  sector  throughput  and 
flight  distance  when  controllers  used  a  combined  voice  and  Data  Link 
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communications  system.  Comparisons  between  the  historical  baseline  and 
the  simulation  results  were  used  to  compute  associated  tangible  benefits. 

In  both  experiments,  additional  test  runs  were  completed  with  increased 
traffic  levels  representative  of  future  demand.  These  data  were  used  in 
conjunction  with  estimates  generated  by  a  fast-time  simulation  model  to 
assess  the  impact  of  Data  Link  in  comparison  to  retaining  a  voice-only 
system  as  traffic  volumes  grow.  Furthermore,  in  both  experiments, 
objective  measures  as  well  as  controller,  supervisor  and  pilot  evaluations 
were  used  to  insure  that  any  measured  benefits  were  achieved  within 
acceptable  limits  of  safe  ATC  and  aircraft  operations. 

An  effort  was  made  to  generalize  the  results  of  the  study  by  determining 
the  incidence  and  nature  of  communications-induced  inefficiencies  in 
domestic  en  route  airspace.  Sector  saturation  problems  similar  to  those 
tested  in  the  Atlanta  ARTCC  were  identified  by  conducting  a  structured 
survey  with  supervisory  personnel  in  ARTCCs  throughout  the  U.S.  These 
data  were  used  to  extend  the  local  study  results  to  an  estimate  of  the 
annualized  national  economic  benefit  that  would  be  achieved  with  Data 
Link  in  congested  en  route  airspace. 

4.  TEST  CONDUCT. 

4.1  TEST  FACILITIES. 

4.1.1  Facilities  Description. 

The  two  real-time  experiments  performed  for  this  study  were  conducted  in 
ATC  simulation  facilities  located  at  the  FAA  Technical  Center.  The 
specific  laboratory  configuration  for  the  study  included  three  key 
components  of  the  FAA  Technical  Center  facilities:  the  NAS  en  route 
laboratory;  the  Target  Generation  Facility  (TGF);  and  the  Data  Link 
laboratory  (figure  1). 

The  NAS  en  route  laboratory  contains  the  Host  computer  system  used  for 
NAS  enroute  data  and  radar  processing.  The  Host  communicates  with 
several  suites  of  the  Plan  View  Display  (PVD)  controller  workstations  that 
are  used  to  display  radar  and  system  data  and  to  enter  system  inputs.  The 
laboratory  appearance  is  identical  to  an  operational  en  route  control  area 
and  includes  a  full  voice  communications  system  and  flight  strip  printers. 

Current  Atlanta  ARTCC  field-release  software  was  implemented  on  the 
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FIGURE  1.  FAA  TECHNICAL  CENTER  SIMULATION  FACILITY 
CONFIGURATION  FOR  DATA  LINK  BENEFITS 
STUDY  EXPERIMENTS 

NAS  laboratory  Host  computer  for  the  present  experiments.  Minimal 
software  modifications  were  completed  to  accommodate  Data  Link 
communications.  SAR  tapes  identical  to  those  normally  generated  by  the 
Host  computer  during  field  operations  were  used  to  collect  data  on  aircraft 
position  and  time  variables.  They  also  permitted  analysis  of  Data  Link 
inputs  and  of  any  aircraft  separation  violations. 

The  NAS  laboratory  is  linked  to  the  TGF  through  the  Host  computer.  The 
TGF  permits  the  Host  and  the  PVD  workstations  to  act  as  a  functioning 
control  facility  by  providing  simulated  radar  data  and  a  means  for 
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interaction  between  aircraft  and  controllers.  The  TGF  includes  a 
laboratory  in  which  pseudopilots  operating  from  specialized  computer 
terminals  can  carry  out  voice  and  Data  Link  communications  with 
controllers  and  make  inputs  to  realistically  maneuver  aircraft  in  response 
to  ATC  clearances.  In  addition,  the  TGF  can  be  linked  to  remotely  located, 
full  fidelity  aircraft  flight  simulators  in  order  to  permit  certified  pilots  to 
participate  in  air  traffic  scenarios. 

TGF  pseudopilots  represented  a  majority  of  the  aircraft  participating  in  the 
test  scenarios  for  the  present  study.  However,  six  flight  simulators  flown 
by  professional  pilots  also  were  used  during  testing.  These  included  two 
B727  simulators  remotely  located  at  Avia  Research  in  California,  and  at  the 
FAA  Mike  Monroney  Aeronautical  Center  in  Oklahoma  (OKC),  and  two 
B747  simulators  located  at  the  NASA  Ames  Research  Center  in  California, 
and  at  the  National  Research  Laboratory  (NLR)  in  the  Netherlands.  The 
final  two  simulators  were  a  General  Aviation  Trainer  (GAT)  and  the 
Reconfigurable  Cockpit  Simulator  (RCS),  both  situated  locally  at  the  FAA 
Technical  Center  in  New  Jersey. 

The  Data  Link  laboratory  houses  a  VAX  11/750  computer  which  acts  as  an 
emulation  of  the  future  ground  Data  Link  processor.  The  VAX  supported 
all  Data  Link  communications  among  controllers  in  the  NAS  en  route 
laboratory,  TGF  pseudopilots,  and  aircraft  flight  simulators. 

4. 1 .2  Facility  and  Scenario  Validation. 

The  primary  function  of  the  FAA  Technical  Center  simulation  facilities  is 
to  support  the  analysis  and  testing  of  ATC  problems  that  arise  during 
ongoing  field  operations.  Because  of  this,  the  test  hardware  and  software 
are  identical  to  those  currently  used  in  NAS  operations.  For  the  purposes 
of  the  present  study,  this  direct  emulation  capability  helped  to  insure  that 
the  ATC  scenarios  and  controller  interfaces  used  in  laboratory  testing 
would  recreate  the  conditions  and  controller  work  environment  which 
existed  on  the  baseline  sample  days  in  the  Atlanta  ARTCC. 

The  Atlanta  ARTCC  supplied  the  SAR  and  voice  radio  tapes  for  the  two 
test  sectors  that  were  used  to  create  the  baseline  test  scenarios.  The 
ARTCC  was  asked  to  provide  a  single  sample  day  tape  for  each  experiment 
that  would  provide  a  representative  example  of  the  daily  rush  period 
problems  experienced  by  the  two  sectors.  The  ARTCC  was  specifically 
requested  not  to  select  sample  days  during  which  weather  or  atypical 
events  complicated  the  ATC  environment. 
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The  sample  day  that  was  provided  for  both  sectors  was  April  18,  1994. 

The  call  signs,  flight  plans  and  types  of  aircraft  that  were  used  in  the  test 
scenarios  for  the  two  experiments  were  identical  to  those  recorded  on  the 
sample  day  SAR  tapes.  Winds  aloft  in  the  simulation  test  scenarios  were 
matched  to  the  sample  day  based  on  records  provided  by  the  National 
Oceanic  and  Atmospheric  Administration  (NOAA)  for  the  Athens,  Georgia, 
reading  on  that  date. 

Because  the  validity  of  the  data  collected  in  this  study  depended  upon 
direct  comparability  of  the  operational  sectors  and  the  test  environment, 
personnel  from  the  Atlanta  ARTCC  were  asked  to  evaluate  the  fidelity  of 
the  simulation.  As  described  in  section  4.4.2,  prior  to  the  study,  two 
controllers  who  were  qualified  at  the  test  sectors  and  who  worked  as  area 
airspace  and  procedures  specialists  participated  in  exercises  intended  to 
develop  preliminary  communications  procedures  for  the  test  scenarios. 
During  these  exercises,  the  controllers  examined  sector  equipment  layouts 
and  controlled  traffic  in  the  test  scenarios.  At  the  conclusion  of  the  effort, 
they  were  asked  to  provide  written  input  regarding  required  changes  to  the 
hardware  configuration,  the  sector  adaptation,  and  any  other  factors  which 
may  have  affected  the  quality  of  the  simulation. 

Both  controllers  indicated  that  the  scenario  traffic  was  typical  for  the  rush 
periods  at  the  two  test  sectors.  In  addition,  they  certified  that  the 
simulation  contained  no  artifacts  which  would  reduce  controller  workload 
in  comparison  to  the  actual  operational  sectors.  The  physical  layouts  of 
the  sector  workstations  also  were  judged  to  be  identical  to  the  operational 
sectors.  The  single  required  change  identified  by  the  Atlanta  procedures 
specialists  was  the  repositioning  of  a  keyboard  at  one  of  the  test  sectors. 
This  modification  was  made  prior  to  the  start  of  the  study. 

Additional  data  confirming  the  validity  of  comparisons  between  the 
simulation  data  and  that  recorded  at  the  operational  facility  were  collected 
during  debriefing  sessions  with  the  test  controllers  and  their  supervisors. 
These  results  are  presented  in  section  5.4.1  of  this  report. 

4.2  DATA  LINK  PERFORMANCE,  SERVICES  AND  AIRCRAFT 
EQUIPAGE. 

The  Data  Link  message  transmission  times  for  these  experiments  were 
controlled  by  the  Data  Link  laboratory  equipment.  This  equipment 
emulated  the  temporal  characteristics  of  Data  Link  communications 
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implemented  on  a  Mode  Select  (Mode  S)  radar  system.  Because  Mode  S  is 
a  narrow  beam  radar,  uplink  and  downlink  times  are  determined  by  the 
rotation  speed  of  the  antenna  and  the  relative  position  of  the  aircraft.  For 
the  present  study,  a  6-second  scan  period  was  assumed.  The  Data  Link 
laboratory  simulated  this  configuration  by  randomly  selecting  uplink 
delays  from  a  rectangular  distribution  ranging  from  0  to  6  seconds.  For 
downlinks.  Mode  S  delays  are  partially  dependent  upon  the  latency  of  the 
aircrew’s  response  to  the  uplink.  If  the  response  time  is  less  than  the  time 
needed  for  a  single  scan,  the  average  downlink  delay  will  be  equal  to  the 
scan  period  of  6  seconds.  However,  if  pilot  response  delays  are  longer, 
downlink  delays  will  increase  in  even  multiples  of  the  scan  period.  Thus, 
for  the  present  study,  the  Data  Link  laboratory  equipment  returned 
downlink  responses  entered  within  6  seconds  of  uplink  receipt  with  a  6- 
second  delay,  responses  entered  within  12  seconds  with  a  12-second  delay, 
etc. 

While  estimated  Mode  S  data  transmission  characteristics  were  assumed 
for  this  study,  the  findings  of  the  experiments  should  be  valid  for  any 
medium  that  would  produce  a  range  of  system  and  human  response  delays 
simular  to  that  experienced  by  the  subject  controllers.  For  purposes  of 
comparison  to  other  Data  Link  transmission  media,  and  to  permit 
identification  of  the  actual  range  of  response  delays  under  which  results 
were  collected  for  this  study,  total  transaction  time  data  were  collected  for 
each  Data  Link  transaction  initiated  during  testing.  Total  transaction  time 
measurements  included  uplink  and  downlink  delays  as  well  as  pilot 
response  latencies.  These  data  are  presented  in  section  5.3.3. 

During  testing,  pilots  and  controllers  had  the  capability  to  employ  a  range 
of  two-way  Data  Link  ATC  communications.  The  message  set  conformed 
to  the  standards  defined  in  DO-219  (RTCA,  1993).  The  set  included 
transfer  of  communication  messages,  as  well  as  altitude,  speed,  and 
heading  clearances.  Initial  contact  messages  were  downlinked  from  the 
equipped  aircraft,  and  unconstrained,  free  text  uplink  messages  were 
available  for  use. 

While  they  are  specified  in  the  DO-219  message  set,  downlinks  of  pilot 
requests,  (e.g.,  requests  for  altitude  or  route  changes)  were  not  available 
for  use  during  the  simulation.  Because  such  aircrew  requests  are  very 
uncommon  in  the  arrival  and  departure  scenarios  that  were  tested  in  this 
study,  and  did  not  appear  in  transcripts  of  the  voice  recordings  for  the 
historical  sample  days  at  the  Atlanta  ARTCC,  the  absence  of  this 
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downlink  capability  was  expected  to  have  minimal  impact  on  the  study 
outcome. 

Approximately  90  percent  of  the  aircraft  in  each  test  scenario  were 
equipped  to  conduct  Data  Link  ATC  communications  in  addition  to  voice 
radio  communications.  To  the  extent  possible,  the  90-percent  equipage 
level  was  achieved  by  assigning  Data  Link  to  the  newest  commercial 
carrier  and  business  aircraft  from  the  sample  day  SAR  tapes  in  order  to 
realistically  portray  likely  future  equipage  schedules. 

4.3  TEST  SUBJECTS. 

The  subjects  for  the  two  experiments  were  18  Full  Performance  Level 
(FPL)  en  route  ATC  specialists  recruited  from  the  Atlanta  ARTCC  who 
regularly  worked  at  the  study  test  sectors.  The  nine  subjects  for 
experiment  1  were  controllers  from  a  high  altitude  en  route  departure 
sector  (Sector  32,  Area  2).  The  nine  subjects  for  experiment  2  were 
controllers  from  an  en  route  arrival  sector  (Sector  09,  Area  5).  In 
accordance  with  current  labor  agreements,  all  subjects  were  selected  for 
participation  in  the  study  by  local  Atlanta  representatives  of  the  National 
Air  Traffic  Controllers  Association. 

Three  controllers  staffed  each  of  the  test  sectors  during  the  rush  periods 
on  the  historic2d  sample  days  that  were  used  as  the  baselines  for  testing. 
For  this  reason,  the  subjects  in  both  experiments  participated  in  teams  of 
three  controllers.  Control  position  assignments  were  made  by  mutual 
agreement  of  each  team  with  one  controller  at  the  radar  (R)  position,  one 
at  the  data  (D)  position,  and  the  third  at  the  tracker  (T)  position.  The 
controllers  maintained  their  position  assignments  throughout  the  test 
trials. 

The  two  experiments  were  conducted  simultaneously  with  one  team  from 
each  sector  participating  in  each  of  three  replications.  The  replications 
were  completed  over  three  consecutive  2-week  study  periods.  Each  of  the 
control  teams  was  accompanied  by  a  different  supervisor  from  its  Area  of 
the  Atlanta  ARTCC. 

4.4  TRAINING  AND  PROCEDURES  DEVELOPMENT. 

In  order  to  maximize  the  validity  of  the  performance  data  collected  in  this 
experiment,  extensive  efforts  were  made  to  accurately  simulate  the 
working  environment  and  conditions  of  an  operational  ATC  facility.  As 


15 


discussed  in  previous  sections,  these  efforts  included  the  use  of  actual  NAS 
equipment  and  the  selection  of  subject  controllers  who  were  thoroughly 
familiar  with  the  test  airspace.  They  also  included  an  attempt  to  provide 
the  subject  controllers  with  a  knowledge  of  Data  Link  and  a  level  of 
proficiency  in  using  it  that  they  would  be  expected  to  have  in  an 
operational  facility  where  Data  Link  had  been  recently  implemented. 

Thus,  subjects  received  extensive  training  and  practice  in  Data  Link 
operation. 

4.4.1  Training. 

The  subjects  were  tr2iined  in  the  use  of  the  Data  Link  system  in  classroom 
sessions  smd  participated  in  team  practice  sessions  in  the  en  route  NAS 
laboratory.  Training  took  place  over  the  first  4  days  of  each  study  session. 
The  first  day  was  devoted  to  briefings  which  described  the  Data  Link 
displays  and  inputs.  These  were  followed  by  an  initial  laboratory  practice 
session  during  which  facilitators  were  available  at  each  sector.  The  second 
and  third  days  of  training  consisted  of  laboratory  practice  sessions  in 
which  traffic  levels  were  raised  progressively  from  40  to  80  percent  of  that 
experienced  on  the  historical  baseline  days.  The  final  training  day  included 
practice  at  traffic  levels  equal  to  the  baseline  days,  as  well  as  instruction  on 
the  rating  and  questionnaire  instruments  that  were  used  during  testing. 

A  Data  Link  training  manual  was  provided  to  the  subjects  for  use  in 
classroom  sessions  (FAA  Technical  Center,  1995).  Laboratory  training 
days  began  with  a  group  discussion  during  which  any  questions  on  Data 
Link  operation  were  addressed.  Each  controller  team  received  an  average 
of  17  hours  of  laboratory  practice  prior  to  the  start  of  data  collection. 

All  simulated  aircraft  were  equipped  with  Data  Link  during  early  training 
sessions  to  focus  on  basic  practice  with  the  new  communication  medium. 
Later  sessions  reduced  equipage  to  the  90-percent  level  used  during  testing 
to  permit  practice  in  alternating  between  voice  and  Data  Link  usage. 

The  later  practice  sessions  used  scenarios  which  presented  air  traffic 
demands  that  were  similar  to  the  test  scenarios  in  terms  of  the  general 
pattern  of  arrival  at  the  sector  boundaries,  mix  of  aircraft  types  and  call 
signs.  Thus,  during  the  training  phase  of  the  experiment,  the  controllers 
experienced  demands  representative  of  daily  variations  at  the  test  sectors. 
However,  to  avoid  bias  attributable  to  anticipation  of  a  specific  traffic 
pattern,  the  configurations  of  air  traffic  arriving  at  the  sectors  that  were 
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dictated  by  the  sample  day  SAR  tapes  were  presented  only  during  actual 
testing. 

4.4.2  Data  Link  Procedures. 

As  they  would  in  an  actual  implementation  of  the  system,  the  controllers 
also  participated  in  exercises  to  develop  efficient  procedures  for  applying 
Data  Link  to  the  type  of  air  traffic  encountered  in  their  sectors.  Prior  to 
the  study,  consulting  procedures  specialists  from  the  Atlanta  ARTCC 
assisted  test  personnel  in  developing  preliminary  procedures  for  the 
scenarios  that  included  use  of  Data  Link  by  the  controller  teams.  These 
procedures  prescribed  general  strategies  for  dividing  different  message 
types  between  voice  and  Data  Link  as  well  as  for  the  division  of  Data  Link 
duties  among  the  three  control  positions. 

The  baseline  procedures  were  introduced  to  the  test  subjects  in  a  briefing 
conducted  after  the  second  day  of  training.  They  were  encouraged  to 
evaluate,  refine  and  modify  these  procedures  within  their  control  teams 
during  subsequent  practice  sessions.  Each  team  described  its  finalized 
procedures  during  a  debriefing  held  prior  to  the  last  practice  session.  Data 
on  the  procedures  adopted  by  the  control  teams  are  reported  in  section 
5.3.1. 

4.5  EXPERIMENT  1  -  DEPARTURE  SECTOR. 

4.5.1  Test  Scenario. 

The  test  scenarios  for  experiment  1  were  based  on  a  recent  historical 
sample  of  air  traffic  derived  from  sector  32  (Spartanburg)  in  area  2  of  the 
Atlanta  ARTCC  (figure  2.).  The  primary  task  of  this  high  altitude  (FL  240 
to  FL  290)  en  route  sector  is  to  control  departures  of  jet  aircraft  climbing 
from  the  Atlanta  and  Charlotte  airports.  In  addition,  sector  32  handles 
arrivals  into  Charlotte  and  overflight  traffic.  On  a  daily  basis  during  the 
early  afternoon,  traffic  saturation  in  this  sector  reaches  a  level  which 
necessitates  the  issuance  of  a  20  MIT  restriction  for  Atlanta  departures 
entering  the  sector.  This  situation  creates  significant  ground  departure 
delays  for  all  aircraft  using  the  runway  that  serves  the  affected  sector. 

The  experiment  was  designed  to  determine  whether  the  MIT  restrictions 
could  be  relaxed  with  the  introduction  of  Data  Link  communications.  In 
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the  test  scenarios,  traffic  demand  at  the  boundaries  of  sector  32  was 
accelerated  from  the  original  sample  SAR  tape  record  to  produce  three 
levels  of  reduced  MIT  restriction, 

4,5.2  Testing  Methodology. 

Following  the  training  and  practice  sessions,  each  control  team 
participated  in  five  test  sessions  in  which  both  Data  Link  and  voice  radio 
were  available  for  ATC  communications.  Each  test  session  consisted  of  a 
1-hour  period  of  ATC  simulation  during  which  the  team  controlled  traffic 
in  a  scenario  derived  by  varying  the  MIT  restriction  on  departure  aircraft 
from  the  Atlanta  airport.  Each  team  was  first  tested  using  a  scenario  in 
which  aircraft  arrival  times  and  positions  at  the  sector  boundary  were 
identical  to  those  on  the  sample  day  SAR  tape  in  which  the  normal  20  MIT 
restriction  was  issued.  The  team  then  attempted  the  problem  with  the 
same  air  traffic  arriving  at  the  sector  boundary  at  times  corresponding  to 
reduced  restrictions  of  15  MIT,  10  MIT,  and  a  test  with  no  restriction 
(minimum  5-mile  separation). 

Boundary  arrival  times,  and  aircraft  sequences  were  calculated  by  a  fast¬ 
time  simulation  of  the  aircraft  queue  at  the  departing  runway  (see  section 
4.8).  In  each  case,  the  sequence  was  determined  by  the  published  gate 
departure  times  for  aircraft  that  had  intended  to  depart  the  runway  on  the 
historical  sample  day  rush  period.  The  total  number  of  aircraft  handled  by 
the  controllers  during  each  of  the  first  4  test  runs  was  41  including  Atlanta 
departures,  Charlotte  departures,  and  overflights.  The  total  number  of 
aircraft  in  the  Atlanta  departure  queue  during  this  period  was  48,  with  24 
of  those  departing  through  sector  32. 

The  fifth  test  run  duplicated  the  traffic  with  no  MIT  restriction.  However, 
to  simulate  future  increases  in  traffic  demand,  a  10-percent  increase  in  the 
number  of  aircraft  departing  the  runway  at  Atlanta  were  injected  into  the 
scenario.  The  additional  aircraft  were  randomly  assigned  to  depart  to 
sector  32  or  to  depart  to  other  en  route  sectors  served  by  the  runway, 
yielding  a  total  of  53  aircraft  in  the  queue,  with  26  departing  through  sector 
32. 

Each  controller  team  and  its  observing  supervisor  participated  in  a 
structured  debriefing  session  after  completing  all  five  test  runs.  The 
debriefing  was  used  to  solicit  comments  from  the  group  regarding  the 
fidelity  of  the  simulation,  procedures  used  to  control  traffic  during  the  test, 
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the  perceived  effectiveness  of  Data  Link,  and  the  Data  Link  human- 
computer  interface. 

4.5.3  Data  Collection  and  Analysis. 

The  primary  benefit  measure  that  was  calculated  for  experiment  1  was  the 
difference  between  the  ground  delay  times  for  departure  aircraft  under  the 
20  MIT  restriction  and  those  that  would  be  experienced  under  the  least 
restrictive  test  run  that  was  safely  completed  during  the  Data  Link 
simulation.  The  ground  delays  at  each  level  of  restriction  were  computed 
using  components  of  the  fast-time  simulation  model  (see  section  4.8). 

Additional  measures  were  collected  during  the  simulation  runs  and 
extracted  from  the  sample  day  SAR,  TGF,  and  voice  recording  tapes  to 
supplement  and  support  the  primary  benefit  calculations.  Measures  of  the 
safety  of  ATC  operations  collected  during  the  test  runs  were  used  as 
criteria  to  determine  the  validity  of  any  performance  benefits  that  were 
observed.  Measures  of  aircraft  throughput  and  other  performance  and 
efficiency  variables  were  applied  to  the  test  and  sample  day  SAR  tapes  and 
to  voice  recordings  as  local  outcome  measures.  In  experiment  1,  these 
outcome  measures  were  used  to  determine  whether  any  ground  delay 
benefit  was  supplemented  by  an  improvement  of  in-flight  efficiency.  Since 
the  criterion  and  outcome  measures  were  used  both  in  experiment  1  and 
experiment  2,  they  are  described  in  detail  in  a  separate  section  of  this 
report  (see  section  4.7). 

4.6  EXPERIMENT  2  -  ARRIVAL  SECTOR. 

4.6.1  Test  Scenario. 

The  scenario  for  this  experiment  was  based  on  a  recent  historical  sample 
of  air  traffic  derived  from  sector  09  (Tiroe)  in  area  5  of  the  Atlanta  ARTCC 
(figure  3.).  This  en  route  sector  controls  airspace  from  11,000  feet  to  FL 
230.  Its  primary  task  is  to  sequence  aircraft  arriving  at  the  Atlanta  airport. 
In  addition,  sector  09  handles  overflight  traffic  and  departures.  Arrivals 
enter  the  sector  in  multiple  streams  and  must  be  sequenced  to  depart  the 
sector  in  trail  over  a  single  fix  (Tiroe)  with  jets  at  14,000  feet  and 
turboprops  at  11,000  feet. 
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FIGURE  3.  ARRIVAL  SECTOR  EXPERIMENT  2 
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From  8:00  a.m.  to  11:00  a.m.  and  from  5:00  p.m.  to  6:00  p.m.  daily,  traffic 
saturation  in  this  sector  typically  creates  a  situation  in  which  the 
controllers  cannot  achieve  the  specified  MIT  for  aircraft  exiting  the  sector 
for  arrival  at  the  Atlanta  airport.  The  traffic  volume  often  forces  the  use  of 
a  holding  pattern  and  extensive  use  of  vectoring.  This  situation  delays 
arrivals  at  the  airport  and  results  in  excess  fuel  usage.  Experiment  2  was 
designed  to  determine  whether  the  expansion  of  communications 
capabilities  using  Data  Link  at  the  sector  would  allow  for  more  efficient 
traffic  flow  for  arriving  aircraft. 

4.6.2  Testing  Methodology. 

Following  the  training  and  practice  sessions,  each  team  participated  in  five 
test  sessions  during  which  Data  Link  and  voice  radio  were  available  for 
ATC  communications.  Each  session  consisted  of  a  1-hour  period  of  ATC 
simulation.  During  the  first  test  session,  the  team  controlled  traffic  in  a 
scenario  which  reproduced  the  arrival  and  overflight  traffic  demand 
recorded  on  the  sample  day  morning  rush  period  for  sector  09  at  the 
Atlanta  ARTCC.  A  total  of  40  aircraft  were  handled  during  the  test 
scenario. 

In  the  final  four  test  runs,  aircraft  were  added  to  the  test  scenario  to 
provide  an  estimate  of  the  team’s  capability  to  control  sector  09  traffic  at 
increased  traffic  demand  levels.  The  added  aircraft  increased  Atlanta 
arrival  traffic  demand  by  approximately  10  percent  on  the  second  run,  20 
percent  on  third,  30  percent  on  the  fourth,  and  40  percent  on  the  fifth. 
Specifically,  Atlanta  arrivals  increased  from  23  on  the  baseline  day  to  31  by 
the  fifth  test  run,  producing  a  final  total  count  of  48  aircraft  in  the  scenario. 

In  order  to  increase  demand  within  the  same  time  span  used  for  the  first 
test  session,  aircraft  were  added  in  gaps  between  arrivals  at  the  test  sector 
that  were  present  on  the  sample  day  SAR  tape.  Because  a  restriction  at  an 
arrival  fix  is  determined  by  the  airport  acceptance  rate,  the  MIT 
restrictions  for  arrivals  were  not  modified  to  accommodate  the  increased 
traffic  test  runs.  All  five  test  runs  were  conducted  using  the  MIT 
restrictions  over  the  arrival  fix  that  were  set  by  the  Atlanta  Traffic 
Management  Unit  during  the  historical  baseline  day  rush  period.  These 
restrictions  began  at  15  MIT,  were  reduced  to  10  MIT  after  the  first  20 
minutes  of  the  sample  period,  and  were  completely  removed  during  the 
final  10  minutes. 
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Each  controller  team  and  its  observing  supervisor  participated  in  a 
structured  debriefing  session  after  completing  all  five  test  runs.  The 
debriefing  w^as  used  to  solicit  comments  from  the  group  regarding  the 
fidelity  of  the  simulation,  procedures  used  to  control  traffic  during  the  test, 
the  perceived  effectiveness  of  Data  Link,  and  the  Data  Link  human- 
computer  interface. 

4.6.3  Data  Collection  and  Analysis. 

The  primary  benefit  assessment  for  experiment  2  was  the  efficiency  with 
which  the  aircraft  in  the  test  scenario  were  passed  through  the  arrival 
sector.  Efficiency  was  measured  using  data  extracted  from  the  historical 
baseline  SAR  tape  and  from  the  SAR  tapes  generated  during  Data  Link 
testing.  The  primary  outcome  measures  included  time  in  sector  and 
distance  flown  for  each  aircraft  (see  section  4.7).  A  fast-time  simulation 
model  based  on  voice-only  operational  capabilities  recorded  on  the 
historical  SAR  tape  was  used  to  generate  baseline  comparison  data  for  the 
increased  traffic  test  runs  using  Data  Link.  Additional  measures  were 
collected  during  the  simulation  runs  and  extracted  from  the  sample  day 
SAR  and  voice  recording  tapes  to  supplement  and  support  the  primary 
benefit  calculations.  Measures  of  the  safety  of  ATC  operations  collected 
during  the  test  runs  were  used  as  criteria  to  determine  the  validity  of  any 
performance  benefits  that  were  observed. 

4.7  EXPERIMENTAL  MEASURES. 

4.7.1  Criterion  Measures. 

The  measures  discussed  below  were  used  in  both  experiments  of  this  study  to 
insure  that  any  improvements  in  the  efficiency  of  operations  that  yielded  a 
user  benefit  were  not  achieved  at  any  loss  in  system  safety.  Both  primary 
indicators  of  safety  and  secondary  or  precursor  indications  of  potential  losses 
in  the  margin  of  safety  were  collected. 

4.7.1. 1  Primary  Criterion  Measures. 

Three  groups  of  measures  were  used  as  primary  criteria  for  determining 
whether  the  ATC  activities  and  aircraft  operations  observed  during  a  test  run 
were  accomplished  m  a  safe  fashion.  Failure  to  pass  any  one  of  the  criteria 
resulted  in  rejection  of  the  performance  data  for  a  test  run.  These  measures 
were  selected  as  primary  criteria  because  they  are  essentially  identical  to 
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those  currently  used  in  the  operational  ATC  environment  to  identify  unsafe 
operating  conditions  and  performance  failures. 

Aircraft  Separation  Violations  and  Causal  Analysis 

The  TGF  and  Host  computers  continuously  monitored  aircraft  separation 
during  the  test  runs  to  detect  violations  of  en  route  ATC  separation 
minimums.  Causal  analysis  of  violations  was  possible  through  the  use  of  TGF 
measures  of  the  severity  and  duration  of  a  violation  as  well  as  track  plots  of 
involved  aircraft.  According  to  procedures  established  for  the  study,  test 
personnel,  controller  subjects  and  the  observing  supervisor  were  designated  to 
make  a  group  determination  of  the  cause  of  any  error.  Potential  causes 
included  simulation  pilot  error,  simulation  equipment  failure,  software  failure, 
or  controller  error.  If  the  error  were  attributed  to  an  equipment/software 
artifact  or  a  pseudopilot  error  which  affected  the  performance  outcome,  the 
test  run  would  be  repeated.  However,  if  the  separation  violation  was 
determined  to  have  been  caused  by  a  controller  error,  the  test  run  would  be 
judged  invalid. 

Supervisory  Safety  Evaluation 

When  working  in  the  en  route  ATC  control  room,  area  supervisors  who  are 
familiar  with  all  aspects  of  sector  operation  monitor  controller  actions  and  air 
traffic  activity  to  ensure  operational  safety.  The  supervisor  uses  expert 
judgment  to  make  determinations  of  potentially  unsafe  conditions  and  to  take 
measures  to  correct  the  problem. 

This  operational  assessment  technique  was  employed  in  the  present  study  in 
the  form  of  a  comprehensive  judgment  for  each  test  run.  The  supervisor 
accompanying  each  team  indicated  his  judgment  after  observing  the  test  run 
by  completing  the  final  item  on  the  supervisory  questionnaire  (see  appendix 
A).  If  a  test  run  was  judged  unsafe,  the  supervisor  was  asked  to  record 
explanatory  remarks. 

Controller  Safety  Evaluation 

In  addition  to  supervisory  monitoring,  it  is  common  practice  in  field  ATC 
operations  for  controllers  to  identify  potentially  unsafe  conditions  that  they 
encoimter  while  staffing  a  sector.  This  evaluation  was  formalized  for  the 
present  experiments  using  a  comprehensive  controller  safety  judgment 
corresponding  to  the  overall  supervisor’s  judgment.  Following  a  test  run,  each 
controller  on  the  participating  team  completed  a  safety  evaluation  (see 
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appendix  A).  If  any  of  the  team  members  judged  the  run  as  unsafe,  they  were 
asked  to  record  explanatory  remarks. 

Pilot  Safety  Evaluation 

Several  aircraft  in  each  test  scenario  were  represented  by  high  fidelity,  piloted 
flight  simulators.  The  professional  aircrew  flying  these  simulators  were  asked 
to  provide  a  comprehensive  evaluation  of  operational  safety  for  each  test  run 
sircar  to  those  solicited  from  the  supervisors  and  controllers  (appendix  A). 
Since  the  aircrew  did  not  have  access  to  a  traffic  display  of  the  sectors  during 
testing,  this  evaluation  was  based  on  the  perceived  accuracy  and  timeliness  of 
communications  with  ATC. 

4.7.1.2  Secondary  Criterion  Measures. 

This  group  of  measures  included  a  number  of  indices  that  are  believed  to  be 
capable  of  sensitively  detecting  controller  states  of  high  workload  or  task 
induced  stress.  Thus,  they  are  indirect  measures  which  may  be  potential 
precursors  to  safety  related  performance  failures.  While  important,  these 
measures  were  not  used  as  criteria  for  accepting  the  validity  of  test  run  data 
since  they  did  not  indicate  performance  failures,  in  and  of  themselves.  Instead 
they  were  intended  as  analytical  aids  for  identifying  potential  changes  in  the 
margin  of  safety, 

Handoff  Qffer/Accept  Measures 

Research  on  cognitive  load  suggests  that  operators  performing  complex  tasks 
often  respond  to  increasing  task  demands  by  sacrificing  performance  on 
secondary  activities.  The  rationale  for  including  the  measures  described 
below  was  that  a  busy  controller  may  begin  to  shed  task  load  by  acceptmg 
handoffs  later  than  normal,  and/or  by  offering  handoffs  earlier,  or  later,  than 
normal.  Each  of  the  parameters  were  measured  objectively  by  analyzmg  the 
baseline  day  and  test  run  SAR  tapes  to  detect  and  record  handoff  offer  and 
accept  times  for  all  aircraft  in  the  scenarios. 

The  following  measure  was  used  to  assess  handoff  acceptance  by  the 
controller  teams; 

Offer-Accept  (O-A)  Interval  --  For  aircraft  entering  the  sector,  the  0-A 
interval  was  defined  as  the  elapsed  time  from  the  sending  sector's  handoff 
offer  keyboard  entry  to  the  receiving  sector's  handoff  accept  entry. 
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The  following  measures  were  used  to  assess  handoff  offers  by  the  controller 
teams.  In  addition  to  identifying  early  handoff  offers  as  a  measure  of  load 
shedding,  these  measures  were  used  in  an  attempt  to  capture  late  handoff 
offers  that  may  have  occurred  because  of  inattention  or  perceptual  overload. 

Offer-Exit  (0-E)  Interval  ~  For  aircraft  leaving  the  sector,  the  O-E 
Interval  was  defined  as  the  elapsed  time  from  the  offer  of  a  handoff  to  the 
ghost  controller  to  the  time  at  which  the  aircraft  crossed  the  exit  boundary . 

O-E  Distance  Flown  --  For  aircraft  leaving  the  sector,  the  O-E  Distance 
Flown  was  defined  as  the  distance  flown  in  tenths  of  miles  during  the  O-E 
Interval. 

Because  these  objective  measures  could  be  confounded  in  cases  where 
handoff  accept  and  offer  times  are  not  related  to  controller  workload,  a 
supervisory  subjective  rating  also  was  used  to  assess  the  timeliness  with 
which  handoffs  were  offered  and  accepted  (see  below). 

Supervisory  Ratings  of  Workload/Performance  Factors 

Following  each  test  run,  the  observing  supervisors  in  these  two  experiments 
were  probed  on  several  factors  believed  to  underlie  their  expert  evaluations  of 
workload  in  the  field.  Because  of  numerous  mitigating  factors,  subjective 
evaluations  were  used  to  measure  these  variables  with  the  supervisor  acting 
as  an  "expert  filter.”  In  questions  preceding  the  overall  safety  judgment  on  the 
questionnaire  described  in  section  4.7. 1.1,  the  supervisors  were  asked  to  judge 
the  extent  to  which  each  of  the  following  events  occurred  during  the  run: 

Errors  or  Omissions  in  Required  Flight  Strip  Marking 

Descending  Arrival  Aircraft  Early  Rather  than  Permitting  Fuel  Efficient 
Descents 

Climbing  Departure  Aircraft  Late  Rather  than  Permitting  Fuel  Efficient 
Climbs 

Untimely  Issuance  of  Clearance 

Failure  to  Comply  with  Letters  of  Agreement 

Early  Handoff  Offers 
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Late  Handoff  Offers 


Delayed  Handoff  Acceptance 
Failure  to  Meet  MIT  Restrictions 

For  each  of  the  above,  the  questionnaire  permitted  the  supervisor  to  indicate 
that  the  event  (1)  never  occurred,  (2)  rarely  occurred,  (3)  occurred,  but  within 
normal  limits  of  operational  acceptability,  (4)  occurred  more  often  than 
normal  for  this  sector  at  this  time  of  day,  or  (5)  occurred  unacceptably  often . 
They  were  also  asked  to  comment  on  perceived  causes  for  any  negative 
judgment. 

Controller  Workload  Ratings 

Controller  workload  was  assessed  using  a  perceived  subjective  workload 
rating  scale.  Following  each  test  run,  each  controller  on  a  team  was  asked  to 
rate  the  previous  work  period  in  comparison  to  a  corresponding  work  period 
at  their  position  in  the  sector  at  the  Atlanta  ARTCC.  The  test  run  workload 
could  be  rated  as  (1)  much  lower  than  usual,  (2)  somewhat  lower  than  usual, 
(3)  about  the  same,  (4)  somewhat  higher  than  usual,  or  (5)  much  higher  than 
usual. 

4.7.2  Outcome  Measures. 

This  class  of  measures  was  used  to  assess  variations  in  operating  efficiency  for 
aircraft  in  flight.  In  the  arrival  experiment  (experiment  2),  these  measures 
were  the  primary  focus  of  the  benefit  assessment.  In  the  case  of  departures 
(experiment  1),  these  measures  did  not  assess  the  target  benefit  of  decreasing 
ground  delays,  but  were  collected  to  identify  any  collateral  in-flight  benefit 
that  may  have  been  associated  with  the  use  of  Data  Link. 

4.7.2. 1  Primary  Outcome  Measures. 

Primary  outcome  measures  were  acquired  by  direct  analysis  of  the  baseline 
and  test  SAR  tapes.  In  each  case,  they  were  used  to  assess  the  overall 
efficiency  with  which  aircraft  operated  within  the  test  airspace  in  terms  of 
time,  distance  traveled,  and  related  resource  usage. 

Time  in  Sector  and  Distance  Flown 

For  the  following  measures,  sector  entry  and  exit  points  were  defined  in 
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terms  of  the  horizontal  and  vertical  boundaries  of  each  test  sector.  If  an 
aircraft  penetrated  the  sector  airspace  after  crossing  the  horizontal 
boundary,  the  entry  point  was  defined  as  the  point  at  which  it  entered  the 
sector’s  altitude  limits.  If  the  aircraft  departed  the  sector  airspace  before 
crossing  the  horizontal  boundary,  the  exit  point  was  defined  as  the  point  at 
which  it  exited  the  sector’s  altitude  limits. 

Aircraft  Time  In  Sector  --  For  each  target  aircraft,  the  elapsed  time  in 
seconds  was  recorded  from  entry  to  the  sector  to  exit  from  the  sector. 

Aircraft  Distance  Flown  in  Sector  -  For  each  aircraft,  the  distance 
flown  in  miles  was  recorded  from  the  sector  entry  point  to  the  exit  point. 

Problem  Exit  Time 


The  problem  exit  time  measure  was  used  to  obtain  an  overall  assessment  of 
the  efficiency  with  which  the  aircraft  in  the  scenarios  traversed  the  test 
airspace.  Problem  exit  time  was  defined  as  the  clock  time  at  which  each 
aircraft  crossed  a  position  in  space  which  marked  the  logical  termination  of 
the  ATC  problem  that  was  studied  in  the  two  experiments.  This  measure  was 
obtained  for  the  original  sample  day  data,  and  for  each  Data  Link  test  run. 

In  experiment  1,  the  aircraft  entered  sector  32  in  two  departure  streams  and 
diverged  within  the  sector  on  routes  to  their  destinations.  For  these  aircraft, 
the  problem  exit  time  was  defined  as  the  clock  time  at  which  each  aircraft 
departed  the  sector  altitude  or  horizontal  boundary. 

In  experiment  2,  all  cirriving  aircraft  routes  converged  at  a  fix  in  sector  09 
(Tiroe)  and  formed  a  stream  of  traffic  for  entrance  to  the  terminal  airspace. 
For  these  aircraft,  the  problem  exit  time  was  defined  as  the  clock  time  at 
which  each  aircraft  crossed  the  Tiroe  fix. 

Since  problem  start  times  were  identical  for  the  sample  day  and  the  initial  test 
runs,  comparisons  of  problem  exit  times  were  used  to  assess  differences  in  the 
effectiveness  with  which  the  total  air  traffic  sample  departed  Atlanta 
(experiment  1)  and  were  delivered  to  the  arrival  fix  (experiment  2). 

Analysis  of  Flight  Path  Records 

The  time  and  distance  measures  discussed  above  could  not  reveal  control 
strategies  that  were  used  to  achieve  specific  time  and  distance  results.  For 
this  reason,  data  visualization  tools  were  applied  to  the  SAR  tapes  to  produce 
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horizontal  and  vertical  profile  tracks  for  the  test  runs  and  for  the  baseline 
days.  These  plots  were  used  to  detect  shortcut  paths  used  by  controllers  , 
holding  patterns  and  vectoring,  and  descent  and  climb  profiles  that  were 
relatively  more  or  less  efficient. 

Fuel  Consumption 

Fuel  consumption  computations  are  strongly  affected  by  preferred  flight 
profiles  and  unique  aircraft  equipment  features  that  differ  among  commercial 
carriers.  Because  of  this,  the  present  experiment  focused  on  flight  time  and 
distance  metrics  to  assess  efficiency.  However,  to  provide  some  indication  of 
how  these  findings  translated  to  resource  usage,  sample  fuel  bum  data  were 
computed  for  selected  aircraft  in  experiment  2  which  were  represented  by 
piloted,  full  fidelity  flight  simulators  and  their  counterpart  aircraft  represented 
on  the  sample  day  SAR  tapes.  The  estimates  for  the  test  mns  were  calculated 
by  fuel  consumption  models  associated  with  the  piloted  simulators.  Estimates 
for  the  historical  baseline  day  flights  were  obtained  from  additional  off  line 
mns  in  which  the  simulators  flew  profiles  that  duplicated  those  of  their 
counterpart  aircraft  on  the  historical  SAR  tapes. 

4.7.2.2  Secondary  Outcome  Measures. 

As  in  the  criterion  measures  category,  in  addition  to  direct  measures  of 
performance,  indicators  of  precursors  to  the  benefit  measures  of  interest  were 
collected  as  a  secondary  outcome  indicator.  This  study  was  based  on  the 
premise  that  adding  Data  Link  to  the  existing  voice  radio  would  expand  the 
control  team's  communications  capacity,  thereby  permitting  them  to  more 
efficiently  handle  aircraft  in  sectors  saturated  because  of  communications 
problems.  Thus,  any  benefit  that  was  measured  should  have  been  correlated 
with  a  redistribution  of  ATC  communications  among  the  two  channels.  The 
following  measures  were  used  to  determine  the  manner  in  which  the  test 
controllers  used  Data  Link  and  voice  to  accomplish  their  communications 
tasks. 

Number,  Duration  and  Content  of  Voice  Messages 

The  frequency  with  which  the  test  controllers  and  simulation  pilots  used  the 
voice  radio  channel  was  measured  by  counting  the  number  of  messages 
contained  in  transcripts  of  the  voice  recordings  made  during  the  test  runs 
and  those  obtained  from  the  Atlanta  ARTCC  for  the  historical  baseline  days. 
The  time  spent  by  the  controllers  and  simulation  pilots  communicating  on  the 
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voice  channel  was  measured  by  manual  timing  of  the  messages  on  the  test 
and  sample  day  audio  recordings. 

The  content  of  the  voice  messages  sent  by  controllers  during  the  test  runs  and 
on  the  sample  days  also  was  assessed  by  reviewing  the  voice  tape  transcripts. 
Messages  were  tabulated  by  message  category.  These  categories  included:  (1) 
individual  speed,  heading  and  altitude  clearances;  (2)  combined  clearances  (3) 
transfer  of  communication  messages;  (4)  responses  to  initial  contact  calls;  (5) 
informational  messages;  (6)  route  changes;  and  other  messages,  as  required. 

Number  and  Content  of  Data  Link  Uplinks  Sent 

The  number  and  content  of  Data  link  messages  sent  by  controllers  were 
assessed  by  analyzing  the  SAR  tapes  from  the  test  runs.  Messages  were 
tabulated  by  message  category.  These  categories  included:  (1)  individual 
speed,  heading  and  altitude  clearances;  (2)  combined  clearances;  (3)  transfer 
of  communication  messages;  (4)  informational  messages;  (5)  route  changes; 
and  other  messages,  as  required. 

4.7.3  Subsidiairv  Measures. 

Two  additional  measures  were  used  to  meet  subsidiary  human  factors 
objectives  of  this  study.  Following  each  test  run,  supervisors  at  each  sector 
were  asked  to  report  on  errors  in  Data  Link  keyboard  inputs.  If  any  errors 
were  observed,  the  supervisors  were  asked  to  describe  how  the  errors  were 
resolved.  They  also  made  a  projective  evaluation  of  the  potential  for 
occurrence  of  undetected  Data  Link  errors  in  a  field  implementation  (see 
appendix  A). 

In  order  to  obtain  information  on  how  sector  tasking  and  communications 
were  distributed  among  the  sector  personnel,  each  controller  on  a  team 
completed  a  duty  profile  after  each  test  run.  The  profile  required  the 
controllers  to  rate  the  frequency  with  which  they  performed  13  sector  tasks 
during  the  previous  test  run  (see  appendix  A). 

4.8  FAST-TIME  SIMULATION. 

A  fast-time  simulation  was  employed  to  supplement  and  support  the  real-time, 
controller-in-the-loop  testing  methodology  used  in  this  study.  This  section 
describes  the  modeling  tool  that  was  used  to  perform  the  fast-time  simulation 
and  presents  a  summary  of  its  applications  to  the  two  experiments. 
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4.8,1  SIMMOD  Description. 


SIMMOD  is  an  airport  and  airspace  computer  simulation  model.  Since  1978, 
SIMMOD’s  development  has  been  supported  by  the  FAA  as  an  operations 
research  tool.  Its  purpose  is  to  test  proposed  improvements  in  facilities, 
procedures,  and  technologies,  and  to  assess  the  impact  of  projected  growth  on 
air  and  ground  operations. 

SIMMOD  is  a  network  model  in  which  points  in  space  are  defined  as  network 
nodes,  and  links  represent  routes  connecting  the  nodes.  Using  this  form  of 
representation,  SIMMOD  can  track  the  simultaneous  movement  of  individual 
aircraft  from  pushback  and  taxi,  to  take-off,  flight,  and  arrival.  Based  on  time 
and  fuel  bum  calculations,  it  can  produce  a  wide  range  of  data  on  travel  time, 
delay,  and  economic  impact. 

The  simulation  logic  of  SIMMOD  is  described  as  “event-stepped.”  Each 
aircraft  movement  is  checked  against  other  possible  conflicting  aircraft  in  the 
problem  before  that  aircraft  is  allowed  to  proceed.  The  airborne  portion  of  the 
model  maintains  sector  capacities  and  minimum  in  trail  separations  by 
simulating  air  traffic  decisions  to  vector,  speed  up,  slow  down  or  hold  aircraft. 
The  simulation’s  look  ahead  capability  permits  realistic  sequencing  of  traffic 
at  merger  points,  balancing  of  en  route  flow,  and  resetting  of  restrictions. 

As  with  other  modeling  techniques,  the  accuracy  and  validity  of  a  SIMMOD 
application  depend  upon  the  data  inputs  that  it  receives.  Building  a  SIMMOD 
model  to  analyze  operational  costs  associated  with  a  particular  air  traffic 
scenario  requires  three  basic  input  files.  The  airspace  file  defines  the  route 
stmctures  and  governing  airspace  procedures  for  the  section  of  airspace  to  be 
simulated.  The  airport  file  defines  the  stmcture  of  any  mnways,  taxiways  and 
gates  required  by  the  model.  The  event  file  contains  the  schedule  of  fli^t 
operations  for  the  aircraft  participating  in  the  scenario. 

For  the  present  study,  SIMMOD  was  used  in  a  selective  fashion  to  address 
specific  questions  about  the  impact  of  Data  Link  on  flight  departure  delays 
and  on  sector  throughput  in  the  Atlanta  ARTCC  sectors  tested  during  the 
experiments  (LeTech,  1995).  In  all  cases,  the  basic  inputs  to  the  model  were 
derived  from  the  historical  SAR  tapes  for  the  sample  days  in  the  two  sectors. 

4.8.2  Experiment  1  Calculations. 

SIMMOD  supported  experiment  1  by  providing  both  test  scenario  design 
inputs  and  benefit  calculations.  Initially,  SIMMOD  was  used  to  determine 
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sector  32  entry  times  imder  the  MIT  restrictions  that  were  tested  in  the 
experiment.  A  SIMMOD  model  was  developed  to  define  the  sector  entry  time 
for  each  departing  aircraft  recorded  on  the  historical  sample  day  SAR  tape  in 
which  a  20  MIT  restriction  had  been  implemented.  This  model  was  validated 
by  comparison  of  the  model  outputs  to  data  taken  directly  from  the  SAR  tape. 
Model  parameter  adjustments  were  made  to  maximize  comparability  of  the 
output  to  the  historical  data.  SIMMOD  then  was  used  to  generate  sector  entry 
times  for  each  aircraft  under  the  15  MIT  restriction,  10  MIT  restriction,  and  no 
restriction  cases.  These  outputs  were  used  to  create  scenario  demand  files  for 
the  Data  Link  simulation  runs  in  experiment  1. 

The  primary  application  of  SIMMOD  in  the  departure  experiment  was  the 
calculation  of  benefits.  Groimd  delay  for  the  20  MIT  base  case  was  computed 
by  subtracting  the  scheduled  departure  time  for  each  aircraft  from  the  actual 
take  off  time  generated  by  the  validated  model.  Scheduled  gate  departure 
times  (P  times)  were  obtained  from  the  P  times  recorded  on  the  historical  SAR 
tape.  Ground  delays  were  then  computed  for  the  15  MIT,  10  MIT  and  no 
restrictions  cases.  The  ground  delays  began  to  accumulate  as  each  aircraft 
reached  the  departure  queue,  and  did  not  include  normal  push  back  and  taxi 
times.  In  computing  these  delays,  the  model  imposed  the  relevant  restriction 
only  on  aircraft  departing  for  sector  32.  However,  the  delay  estimates 
included  the  effects  that  the  restrictions  have  on  all  aircraft  departing  the 
same  runway  during  the  sample  period.  That  is,  the  total  delay  calculation 
included  the  impact  on  aircraft  entering  sector  32,  and  on  those  whose  take  off 
times  were  affected  by  the  restricted  aircraft.  The  net  benefit  was  computed 
by  comparing  the  total  aircraft  delays  under  the  20  MIT  restriction  and  the 
minimum  level  of  restriction  imder  which  the  subject  controllers  were  able  to 
safely  control  traffic  during  the  manned  simulation  testing. 

SIMMOD  also  was  used  to  assess  the  impact  on  total  delays  of  increasing  the 
departure  traffic  demand.  The  number  of  departing  aircraft  were  increased 
by  10  percent,  and  the  model  was  run  with  the  historical  20  MIT  restriction  to 
calculate  baseline  delays.  The  model  was  then  run  with  no  departure 
restrictions  and  the  10-percent  traffic  increase.  The  delay  difference  between 
the  two  runs  was  used  £is  the  benefit  estimate  if  the  subject  controllers  were 
able  to  safely  control  the  increased  traffic  with  no  restrictions  using  the 
combined  Data  Link  and  voice  radio  communications  during  manned 
simulation. 

4.8.3  Experiment  2  Calculations. 

The  basic  test  for  experiment  2  compared  the  flow  of  traffic  on  the  historical 
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sample  day  to  that  obtained  under  Data  link  communication  conditions  in  the 
real-time  simulation.  This  test  was  completed  by  direct  extraction  of  outcome 
measures  from  the  original  and  experimental  SAR  tapes.  To  support 
additional  benefit  calculations,  SIMMOD  was  used  to  provide  estimates  of 
sector  09  throughput  and  operating  efficiency  under  voice  radio-only 
conditions  with  traffic  levels  increased  from  those  experienced  on  the  sample 
day.  This  output  was  compared  to  live  simulation  data  obtained  when  subject 
controllers  using  a  combined  Data  Link  and  voice  radio  system  attempted  to 
cope  with  additional  increases  in  traffic  volume. 

A  SIMMOD  model  of  sector  09  was  first  constructed  to  emulate  the  traffic 
flow  and  operating  efficiencies  which  characterized  the  performance  recorded 
on  the  historical  SAR  tape.  The  Atlanta  arrival  traffic  demand  at  the  sector  09 
boundary  was  then  increased  on  successive  model  runs  by  10, 20, 30,  and  40 
percent  to  provide  estimates  of  the  throughput  and  efficiency  that  would  be 
demonstrated  if  the  sector  were  challenged  by  these  demands  using  current, 
voice-only  communications.  The  output  was  compared  to  the  data  produced 
by  live  simulation  with  Data  Link  when  traffic  in  the  test  scenarios  was 
increased  in  identical  increments. 

5.  RESULTS. 

5.1  EXPERIMENT  1  -  DEPARTURE  SECTOR. 

As  described  in  section  4,  the  primary  user  benefit  of  Data  Link 
communications  hypothesized  for  experiment  1  was  a  reduction  in  ground 
delay  for  aircraft  awaiting  departure  from  the  runway  serving  sector  32. 
The  magnitude  of  this  benefit  depended  on  whether  the  test  controllers 
could  safely  accept  and  control  the  increased  traffic  density  that  would 
occur  as  the  MIT  restrictions  were  reduced  on  departures  entering  the 
sector.  The  following  subsections  first  present  data  on  the  criterion  safety 
measures  that  were  collected  on  each  of  the  five  test  runs  completed  by  the 
three  teams  of  sector  32  controllers.  Ground  delay  data  are  then  presented 
which  correspond  to  the  MIT  restrictions  at  which  test  runs  were  safely 
accomplished  by  the  subjects. 

5.1.1  Criterion  Performance . 

5. 1 . 1 . 1  Aircreift  Separation. 

Aircraft  separation  was  continuously  monitored  at  the  Host  operator’s 
console  during  each  test  run.  No  operational  errors  (minimum  en  route 
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separation  violations)  were  recorded  between  aircraft  in  sector  32  on  any 
of  the  five  test  runs  for  any  of  the  three  teams  of  controllers. 

5. 1.1.2  Operational  Safety  Assessments. 

Controller  and  Supervisor  Ratings 

Expert  operational  assessments  of  the  safety  with  which  the  test  runs  were 
completed  are  shown  in  figure  4.  The  histogram  presents  the  frequency 
with  which  the  15  test  runs  (five  per  team)  received  each  of  the  four 
possible  safety  ratings  from  the  radar,  data  and  tracker  controllers  as  well 
as  the  observing  supervisor  for  the  sector. 

As  shown  in  the  figure,  none  of  the  controllers  or  supervisors  rated  any  of 
the  test  runs  as  “unsafe.”  Supervisors  assigned  the  test  runs  the  highest 
ratings  with  1 1  runs  rated  as  having  a  greater  margin  of  safety  than  normal 
for  the  sector  in  the  operational  voice-only  environment,  and  Wr  runs 
rated  as  having  typical  levels  of  safety.  The  controllers  rated  the  safety  of 
a  majority  of  the  test  runs  as  typical  for  the  sector  during  the  rush  period 
(31  of  45  ratings).  Eleven  runs  were  rated  as  having  a  higher  margin  of 
safety  than  normal. 
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FIGURE  4.  SECTOR  32  SAFETY  RATINGS 
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Three  of  the  45  controller  ratings  were  in  the  category  “safe,  but  some 
concern.”  Two  of  these  ratings  were  assigned  by  the  same  radar  controller 
to  runs  one  and  two  (20  and  15  MIT).  The  third  was  assigned  by  a  data 
controller  on  run  two  (15  MIT).  Comments  written  by  these  personnel 
indicated  that  they  felt  that  safety  had  not  been  compromised  during  either 
of  the  two  initial  test  runs.  Although  no  input  error  occurred,  they 
expressed  some  concern  over  the  possibility  that  a  keyboard  error  could 
have  caused  a  separation  violation.  Both  controllers  rated  safety  on  the 
three  remaining,  higher  difficulty  test  runs  as  typical  for  the  sector. 

Further  examination  of  the  ratings  across  test  runs  revealed  no  evidence 
for  a  reduction  in  perceived  safety  as  a  function  of  the  complexity  of  the 
control  problem.  On  the  1  (higher  margin  of  safety)  to  4  (unsafe)  scale, 
average  team  ratings  for  the  three  groups  of  controllers  and  supervisors 
were  1.83  for  both  the  baseline  20  MIT  and  15  MIT  spacings,  1.75  for  the  10 
MIT  spacing,  and  1.5  for  both  the  unrestricted  run  with  baseline  traffic 
levels  and  the  unrestricted  run  in  which  departure  traffic  was  increased  by 
10  percent. 

Pilot  Ratings 

Pilot  safety  assessments  were  performed  as  part  of  a  post  run 
questionnaire  completed  by  the  two  pilots  who  flew  each  full  fidelity  flight 
simulator  during  the  test  runs,  and  by  simulation  site  coordinators  who 
acted  as  observers.  They  rated  the  flights  on  a  scale  ranging  from 
“completely  safe”  (1)  to  “completely  unsafe”  (7).  The  FAA  RCS  configured 
as  a  B747,  the  Avia  727,  and  the  NASA  B747  participated  in  all  sector  32 
flights.  The  FAA  and  Avia  simulators  were  equipped  to  receive  Data  Link. 
The  NASA  simulator  was  able  to  communicate  using  voice  radio  only. 

On  each  test  run,  the  FAA  and  Avia  simulators  completed  one  flight,  while 
the  NASA  simulator  completed  two  flights.  Thus,  observer,  pilot  flying 
(PF)  and  pilot  not  flying  (PNF)  safety  ratings  were  obtained  on  four  flights 
for  each  of  five  test  runs.  Over  the  three  replications  of  the  experiment, 
this  produced  a  total  of  60  safety  rating  opportunities. 

For  the  purpose  of  analysis,  the  three  ratings  for  each  flight  were  summed 
to  produce  a  scale  with  a  minimum  value  of  3  (completely  safe)  to  21 
(completely  unsafe).  Of  the  60  flights,  49  (82  percent)  received  combined 
ratings  of  3,  indicating  unanimous  agreement  among  the  raters  that  the 
flight  had  been  completely  safe.  The  remaining  11  flights  all  received 
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ratings  below  the  midpoint  of  the  combined  safety  scale  (12),  with  7  of 
these  flights  receiving  combined  ratings  of  4  or  5. 

The  poorest  individual  ratings  were  assigned  to  2  of  the  60  flights.  On  one 
flight  during  run  5  of  the  experiment,  the  PNF  assigned  a  rating  of  6  on  the 
1  to  7  scale.  The  PF  and  observer  assigned  the  flight  a  2  and  3, 
respectively.  Written  comments  from  all  three  raters  indicated  that  the 
perceived  reduction  in  safety  was  attributable  to  a  failure  of  the  simulated 
Data  Link  avionics.  Both  pilots  noted  that  the  computer  display  unit 
(CDU)  that  was  used  to  display  messages  and  enter  responses  would  not 
accept  message  confirmation  (wilco)  entries,  and  that  it  had  presented 
numerous  error  messages. 

On  a  second  flight  during  run  4,  the  PNF’s  rating  was  5,  while  the  PF  and 
the  observer  each  rated  the  flight  as  “completely  safe”  (1).  The  PNF’s 
comments  indicated  that  he  felt  that  he  had  missed  a  Data  Link  message 
after  receiving  a  radio  inquiry  from  the  controller  regarding  his  failure  to 
respond  to  an  altitude  clearance.  The  observer’s  comments  indicated  that 
the  PNF  had,  in  fact,  responded  to  the  Data  Link  clearance,  but  that  the 
system  had  fedled  to  return  the  response  to  the  controller.  The  observer 
explained  that  it  had  been  the  PNF’s  first  test  run,  and  that  he  had  been 
confused  about  the  meaning  of  an  error  message  from  the  simulated  Data 
Link  equipment  which  had  signaled  the  failure  of  the  downlink  attempt. 

The  incidents  described  above  suggest  that  the  poor  safety  ratings  for  2  of 
the  60  cases  were  caused  by  local  simulation  equipment  failures.  Since 
neither  rating  was  attributable  to  controller  error  or  to  inherent  problems 
with  Data  Link  ATC  communications,  the  associated  test  runs  were  not 
judged  invalid. 

5.1.2  Ground  Delay. 

The  results  discussed  in  section  5.1.1  demonstrate  that  the  control  teams  in 
all  three  replications  of  the  experiment  were  able  to  safely  control  the  air 
traffic  at  all  reduced  spacing  restrictions.  These  included  the  conditions  in 
which  air  traffic  demands  were  increased  and  aircraft  were  permitted  to 
arrive  at  the  sector  with  the  legal  minimum  5-mile  spacing.  Ground  delay 
reductions  associated  with  the  performance  achieved  by  the  control  teams 
using  Data  Link  are  presented  below. 

SIMMOD  airport  calculations  were  performed  for  all  four  spacing 
restrictions  tested  during  the  study  in  order  to  compute  the  ground  delays. 


36 


Total  delay  for  the  48  aircraft  departing  the  runway  serving  sector  32 
under  the  20  MIT  restriction  that  was  in  force  during  the  sample  day  rush 
period  was  1,795  minutes,  or  an  average  of  37.4  minutes  per  aircraft. 
Reductions  in  total  ground  delays  for  the  same  aircraft  sample  when  the 
use  of  Data  Link  permitted  the  relaxation  of  restrictions  are  illustrated  in 
figure  5. 

Because  the  controllers  in  the  present  experiment  were  able  to  reliably  and 
safely  control  the  baseline  traffic  sample  with  the  minimum  5-mile  spacing, 
the  maximum  reduction  in  delay  shown  in  the  figure  was  achieved.  The 
687  minutes  of  delay  with  no  restriction  in  force  represents  an  average 
reduction  of  23.1  minutes  per  aircraft  (62  percent)  in  ground  time  from  the 
daily  20  MIT  restriction  that  is  currently  issued  at  the  Atlanta  ARTCC 
using  voice-only  communications. 

Figure  5  also  presents  the  results  of  SIMMOD  ground  delay  calculations 
when  aircraft  in  the  queue  to  depart  the  runway  serving  sector  32  were 
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FIGURE  5.  ATLANTA  DEPARTURE  GROUND  DELAY  SAVINGS 
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increased  by  10  percent.  As  shown  in  the  figure,  in  comparison  to  the 
baseline  traffic,  the  increase  in  departure  runway  demand  increased  total 
delay  under  all  restrictions.  However,  the  absolute  magnitude  of  the 
increase  in  delay  diminished  with  the  reduction  in  MIT  restrictions.  Under 
the  20  MIT  restriction  normally  enforced  for  sector  32,  a  10-percent  growth 
in  departures  increased  total  delay  by  582  minutes.  However,  at  the 
minimum  5-mile  spacing  that  was  successfully  tested  using  Data  Link,  the 
total  delay  increased  by  only  197  minutes. 

Taken  together,  the  results  suggest  that,  if  Data  Link  were  implemented  to 
eliminate  the  current  20  MIT  restriction,  there  would  be  a  major  reduction 
in  average,  per  aircraft  ground  delay  under  present  demand  conditions. 
Furthermore,  while  future  increases  in  demand  under  the  current  20  MIT 
restriction  would  result  in  a  rapidly  accelerating  increase  in  ground  delays 
at  Atlanta,  they  would  be  accommodated  with  a  much  slower  growth  in 
total  delay  in  the  unrestricted  departure  environment  made  possible  by 
Data  Link  ATC  communications.  Estimates  of  the  economic  benefits 
associated  with  the  ground  delay  savings  are  presented  in  section  6  of  this 
report. 

5.1.3  Sector  Throughput. 

While  the  primary  focus  of  this  experiment  was  Data  Link’s  impact  on 
ground  delays,  measures  of  in-fli^t  performance  also  were  collected.  The 
goals  of  the  following  8malyses  were  to  (1)  determine  whether  Data  Link 
provided  any  collateral  in-flight  benefit,  and  (2)  ensure  that  any  ground 
delay  reduction  was  not  compromised  by  a  loss  in  the  efficiency  with  which 
aircraft  were  controlled  in  the  test  sector. 

Figures  6  and  7  present  the  mean  aircraft  flight  time  and  distance  flown  in 
sector  32.  The  data  shown  in  the  graphs  for  the  voice-only  condition  were 
derived  directly  from  the  sample  day  SAR  tape.  Data  for  Data  Link  testing 
are  averages  across  the  three  test  control  teams.  All  aircraft  that  flew  in 
the  sector  during  the  sample  period  are  represented  in  the  graphs 
including  Atlanta  departures,  Charlotte  departures,  and  overflights. 

As  shown  in  the  figures,  the  addition  of  Data  Link  communications  was 
associated  with  a  reduction  in  mean  aircraft  flight  time  and  distance  over 
all  test  conditions.  The  average  aircraft  on  the  historical  baseline  day  flew 
52  miles  in  the  sector  over  almost  7  minutes.  Collapsing  across  all 
restrictions  and  traffic  levels,  mean  flight  distance  was  reduced  to  40  miles 
over  5.5  minutes  during  Data  Link  testing. 
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In  examining  these  data,  it  should  be  noted  that  the  flight  time  and 
distance  measures  are  closely  correlated.  This  suggests  that  the  controller 
teams  did  not  use  potentially  inefficient  speed  increases  to  reduce  time  in 
sector.  However,  evaluation  of  sample  flight  track  profiles  from  the 
baseline  Atlanta  data  and  the  test  trials  did  reveal  one  identifiable  source 
of  the  observed  improvement  in  sector  throughput. 

While  no  general  differences  in  control  strategy  were  observed,  the  Atlanta 
departure  traffic  appeared  to  be  handled  differently  on  the  baseline  day 
and  in  the  test  runs.  Specifically,  departures  chmbing  into  the  sector  on 
the  baseline  day  when  only  voice  communications  were  available  tended  to 
be  held  at  altitudes  within  the  sector  boundaries  for  a  longer  period  time 
before  departing  the  ceiling  for  their  desired  cruise  altitudes.  During  Data 
Link  testing,  it  appeared  that  controllers  were  able  to  release  these  aircraft 
to  the  higher  altitudes  sooner,  resulting  in  shorter  sector  flight  times  and 
distances. 

Comparison  of  the  data  from  individual  test  conditions  in  figures  6  and  7 
also  demonstrates  that  the  in-flight  benefits  discussed  above  were  not  a 
result  of  the  reduced  MIT  restrictions  that  were  employed  during  testing. 
While  the  fined  four  test  runs  were  completed  under  different  traffic 
density  patterns  for  arrival  at  the  sector  boundary,  the  first  run  precisely 
duplicated  the  pattern  of  traffic  produced  by  the  20  MIT  restriction  on  the 
historical  sample  day.  Thus,  the  time  and  distance  data  for  the  first  run  are 
directly  comparable  to  the  baseline  day.  Both  of  these  average  in-flight 
measures  for  the  first  test  run  show  an  improvement  over  the  baseline  day. 

Figure  8  breaks  out  the  flight  time  and  distance  means  for  the  three 
categories  of  air  traffic  that  flew  in  sector  32  under  the  20  MIT  restriction. 
As  shown  in  the  figure,  the  Atlanta  departures,  which  represented  the 
primary  traffic  flow  during  the  rush  period,  reflect  the  overall  improvement 
in  efficiency  shown  in  the  combined  comparison  of  the  voice-only  baseline 
and  the  Data  Link  test  data.  In  addition,  similar  reductions  are  apparent 
for  the  Charlotte  departures  and  overflight  aircraft  handled  by  the  sector. 
Thus,  the  results  indicate  that  improvements  in  the  primary  traffic  flow  in 
the  sector  were  not  achieved  by  sacrificing  service  to  other  aircraft. 

Traditional  statistical  analysis  of  the  significance  of  the  results  of  the 
throughput  measures  were  not  warranted  because  of  the  relatively  low 
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FIGURE  8.  TIME  AND  DISTANCE  FLOWN  IN  SECTOR  32 
BY  ATLANTA  DEPARTURES,  CHARLOTTE 
DEPARTURES,  AND  OVERFLIGHTS 

number  of  control  teams  that  could  be  practically  tested.  However,  since 
the  three  teams  represent  a  reasonably  large  sample  of  all  controllers  from 
Area  2  of  the  Atlanta  ARTCC,  the  reliability  of  the  findings  can  be  inferred 
by  examining  the  repeatability  of  the  outcome  across  the  three 
independent  test  teams.  Figure  9  displays  the  flight  time  and  distance  data 
averaged  across  all  test  runs  for  each  of  the  control  teams  that  participated 
in  the  experiment.  As  shown  in  the  figure,  all  three  teams  displayed 
improved  efficiency  during  testing  with  the  Data  Link  system  in 
comparison  to  their  counterparts  using  only  voice  radio  on  a  typical  day  at 
the  operational  facility. 

An  alternative  statistical  analysis  of  the  data  was  conducted  by  treating  the 
individual  Atlanta  departure  aircraft  as  subjects,  and  comparing  each 
aircraft’s  flight  efficiency  on  the  baseline  day  to  its  performance  under  the 
three  test  teams’  control  for  the  20  MIT  condition.  The  repeated  measures 
analysis  of  variance  indicated  that  departure  performance  with  Data  Link 
was  significantly  better  than  that  on  the  baseline  day  under  voice-only 
communications  (Fig, 57=  8.68,  p<.01). 
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FIGURE  9.  TIME  AND  DISTANCE  IN  SECTOR  FOR  THE 
THREE  SECTOR  32  TEAMS 

Figure  10  presents  a  final  portrayal  of  the  in-flight  efficiency  data.  The 
histogram  shows  the  processing  time  for  all  Atlanta  departure  aircraft 
averaged  across  teams  for  each  test  condition.  Processing  time  was 
calculated  for  the  baseline  day  and  the  test  conditions  by  subtracting  the 
problem  exit  time  for  the  first  Atlanta  departure  aircraft  leaving  sector  32 
from  that  of  the  last  aircraft.  Thus,  processing  time  reflects  time  needed  to 
complete  the  sector’s  task  of  controlling  the  entire  departure  traffic  sample 
during  the  rush  period  in  order  to  place  the  aircraft  on  their  destination 
routes  at  the  boundaries. 

This  view  of  the  data  demonstrates  that  the  ground  delay  savings  obtained 
at  each  reduced  MIT  restriction  made  possible  by  Data  Link 
communications  were  not  only  preserved  as  the  aircraft  departed  the 
sector,  but  were  supplemented  by  the  enhanced  in-flight  operational 
efficiency.  As  shown  in  the  figure,  47  minutes  were  needed  to  process  all 
of  the  departures  on  the  historiced  baseline  day  using  voice-only 
communications.  Under  testing  where  the  communications  channel  was 
expanded  by  Data  Link,  processing  time  was  reduced  by  an  average  of 
16.6  minutes  (35  percent)  at  the  minimum  spacing  for  the  same  traffic 
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FIGURE  10.  DEPARTURE  TRAFFIC  PROCESSING  TIME  IN  SECTOR  32 

sample.  The  data  also  show  that  departure  processing  time  was  shorter 
even  when  the  aircraft  were  subjected  to  the  same  20  MIT  restriction  as 
that  imposed  on  the  baseline  day  (45.5  minutes),  and  when  two  additional 
aircraft  were  added  to  the  traffic  sample  with  minimum  spacing  (32.8 
minutes). 

Section  6  presents  estimates  of  the  economic  benefits  associated  with  the 
in-flight  efficiency  improvements  discussed  above,  along  with  those 
provided  by  the  ground  delay  savings  obtained  in  this  experiment. 

5.1.4  Secondary  Criterion  Measures. 

As  described  in  section  4,  operational  safety  measures  were  used  as 
criteria  for  validating  the  NAS  user  benefits  identified  during  testing.  The 
findings  discussed  in  this  section  were  obtained  using  a  group  of  measures 
intended  to  determine  whether  these  benefits  were  achieved  at  any 
expense  to  the  general  quality  of  task  performance  exhibited  by  the 
controllers  or  any  increase  in  the  worldoad  levels  that  they  experienced. 
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5. 1 .4. 1  Supervisor  Performance  Ratings. 


During  each  test  run,  the  supervisors  were  asked  to  judge  the  control  team 
on  nine  factors  indicative  of  degraded  sector  performance.  A  5-point  rating 
scale  ranging  from  1  “never  occurred”  to  5  “occurred  unacceptably  often” 
was  used  to  quantify  the  judgments. 

Ratings  on  each  dimension  for  the  five  test  conditions  averaged  across  the 
three  supervisors  are  summarized  in  figure  11.  None  of  the  average 
ratings  exceeded  the  midpoint  on  the  scale  (3  -  “within  normal  limits  for 
the  sector  during  this  duty  period”).  Although  flight  progress  strip 
marking  appeared  to  receive  the  poorest  average  ratings  overall,  none  of 
these  indicated  that  omissions  or  errors  were  greater  than  normal  for  the 
sector  during  the  duty  period  under  examination  in  the  experiment. 
Inspection  of  the  data  for  individual  teams  indicated  that  all  of  the  five 
ratings  of  the  nine  factors  made  by  each  supervisor  were  “within  normal 
limits,”  or  better.  As  shown  in  the  figure,  the  mean  ratings  also  did  not 
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FIGURE  11.  SECTOR  32  SUPERVISOR  RATINGS  OF  CONTROLLER 
PERFORMANCE 
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reveal  any  tendency  for  deterioration  in  sector  performance  as  the  MIT 
restrictions  were  reduced  or  treiffic  was  increased. 

5. 1 .4.2  Controller  Workload. 

Controller  workload  was  assessed  using  a  subjective  rating  scale  of 
perceived  effort.  Following  each  test  run,  the  controllers  rated  their 
workload  on  a  5-point  scale  relative  to  their  normal  workload  at  sector  32 
for  the  same  work  period.  Figure  12  presents  the  mean  workload  ratings 
for  each  control  position  across  the  five  test  runs. 

As  indicated  by  the  figure,  the  mean  ratings  for  all  test  runs  and  control 
positions  were  below  the  midpoint  rating  of  3  (“about  the  same”).  In 
addition,  there  was  no  apparent  trend  for  average  workload  ratings  to 
increase  when  the  MIT  restrictions  were  reduced  or  when  the  traffic 
demand  increased  by  two  Eiircraft. 

None  of  the  individual  subjects  assigned  a  workload  rating  of  5  -  “much 
higher  than  normal”  to  any  of  the  test  runs.  “Somewhat  higher  than 
normal”  (4)  ratings  were  received  by  the  15  MIT  test  run  from  the  radar, 
data  and  tracker  positions  on  one  team,  and  by  the  no  restriction  and  110 
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FIGURE  12.  SECTOR  32  CONTROLLER  WORKLOAD  RATINGS 
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percent  traffic  runs  from  the  radar  position  on  another  team.  Overall 
however,  of  the  45  individual  measures  obtained  over  all  test  runs  and 
teams,  26  ratings  indicated  that  workload  was  “much  lower”  or  “somewhat 
lower”  than  normal,  and  14  ratings  indicated  that  position  workload  was 
“about  the  same”  as  normal  for  the  sector  during  the  tested  work  period. 

5. 1.4.3  Handoff  Measures. 

Measures  of  controller  performance  in  accepting  handoffs  from,  and 
offering  handoffs  to,  adjacent  sectors  were  collected  during  the  experiment 
as  subsidiary  workload  indices.  Figure  13  presents  results  for  two  handoff 
measures  obtained  from  the  baseline  Atlanta  SAR  tape  and  the  comparable 
20  MIT  test  run  with  Data  Link.  As  discussed  below,  both  measures  were 
in  agreement  with  the  subjective  ratings,  indicating  no  significant  change 
in  controller  workload  under  Data  Link  testing. 


The  O-A  Interval  is  an  index  of  the  average  time  taken  by  the  control  team 
to  accept  an  aircraft  handoff  from  a  sending  sector.  It  was  hypothesized 
that  increased  workload  may  have  been  reflected  by  delays  in  accepting 
the  offer.  As  shown  in  figure  13,  no  evidence  of  an  increase  in  controller 
workload  during  Data  Link  testing  was  provided  by  this  measure.  In  fact. 


Offer-Accept  Interval  ^Offer-Exit  distance  to  boundry 


FIGURE  13.  SECTOR  32  HANDOFF  MEASURES 
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handoffs  were  accepted  an  average  38  seconds  earlier  during  testing  than 
they  were  on  the  baseline  day. 

The  O-E  Distance  Flown  is  an  index  of  the  average  distance  from  the 
aircraft  boundary  at  which  the  sector  32  controllers  offered  a  handoff  to 
the  next  sector  along  the  aircraft’s  route  of  flight.  It  was  hypothesized  that 
increased  workload  in  sector  32  may  have  been  evidenced  either  by 
offering  handoffs  earlier  to  shed  load,  or  by  an  inappropriate  delay  in  the 
handoff  caused  by  overload  attributable  to  other  sector  tasks. 

The  results  show  that,  under  Data  Link  testing,  the  controllers  offered 
handoffs  an  average  of  15  miles  closer  to  the  sector  boundary  than  their 
counterparts  did  on  the  historical  baseline  day.  The  mean  flight  distance 
and  time  to  the  exit  boundary  were  approximately  19  miles  and  2.5 
minutes,  in  comparison  to  33  miles  and  4.5  minutes  for  the  baseline  day. 
Both  sets  of  values  are  within  normal  limits  for  offering  handoffs. 

Rather  than  being  indicative  of  increased  workload,  the  relatively  reduced 
distance  to  the  sector  boundary  under  Data  Link  testing  appears  to  have 
been  the  result  of  a  strategy  difference  between  normal  voice  and  Data 
Link  communications  procedures.  A  typical  strategy  when  using  voice 
radio  is  to  hand  off  an  aircraft  to  the  next  sector  well  before  reaching  the 
boundary,  but  to  retain  communications  capability  until  the  controller  is 
certain  that  no  additional  clearances  must  be  sent  to  the  aircraft. 

The  Data  Link  services  provided  the  controllers  with  an  option  to  select  an 
automatic  transfer  of  communication  (TOC)  feature.  When  set  to  the 
automatic  mode,  a  TOC  message  was  prepared  and  sent  without  further 
controller  intervention  when  the  manually  entered  handoff  was  accepted. 
This  automatic  feature  was  widely  used  by  the  teams  in  the  present  study. 

In  order  to  make  effective  use  of  the  automatic  TOC  while  retaining 
communications  with  the  aircraft  for  the  required  period  of  time,  the 
controllers  adopted  a  strategy  of  delaying  the  handoff  offer.  As  shown  in 
the  figure,  this  decreased  the  O-E  Distance.  However,  as  indicated  by  the 
supervisor  ratings  for  handoff  performance  (section  5. 1.4.2)  and  by  the 
acceptable  flight  time  and  distance  to  the  boundary  that  were  achieved,  the 
strategy  had  no  discernible  negative  impact  on  ATC  system  performance. 
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5.1.5  Communications. 


Measures  of  the  nature  and  content  of  communications  conducted  on  the 
baseline  day  and  during  testing  were  obtained  as  secondary  outcome 
measures.  As  discussed  in  section  4.7.2.2,  it  was  expected  that  any  user 
benefit  observed  during  testing  would  be  associated  with  a  redistribution 
of  communications  across  the  voice  and  Data  Link  media. 

5.1.5. 1  Voice  Radio  Usage. 

Figure  14  illustrates  the  dramatic  reduction  in  voice  radio  usage  that 
occurred  during  testing  when  Data  Link  was  available  for  ATC 
communications.  As  shown  in  the  figure,  during  the  rush  period  on  the 
baseline  day  at  the  Atlanta  ARTCC,  the  radio  channel  was  occupied  for 
more  than  16.5  minutes.  During  Data  Link  testing,  radio  channel 
occupation  time  for  the  average  sector  32  control  team  across  all  test  runs 
dropped  by  84  percent  to  less  than  2.6  minutes. 

Detailed  ansdyses  of  the  number  and  content  of  voice  messages  were 

20 
18 
16 
14 
12 
10 
8 
6 
4 
2 
0 


FIGURE  14.  VOICE  RADIO  CHANNEL  OCCUPATION  TIME 
IN  SECTOR  32 
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completed  for  the  historical  baseline  rush  period  in  sector  32  and  the 
comparable  20  MIT  Data  Link  tests.  On  the  baseline  day,  the  controllers 
made  217  voice  transmissions  while  pilots  made  179  transmissions.  During 
Data  Link  testing  at  20  MIT,  the  total  number  of  controller  voice 
transmissions  dropped  to  an  average  of  23.3  while  pilot  transmissions  fell 
to  an  average  of  26.  This  equates  to  an  average  88  percent  reduction  in 
combined  pilot  and  controller  voice  messages. 

Table  1  presents  a  breakdown  of  the  t5q)es  of  voice  messages  sent  by 

TABLE  1.  VOICE  RADIO  MESSAGES  IN  SECTOR  32  ON  HISTORICAL 
BASELINE  AND  DATA  LINK  TEST  DAYS  AT  THE  20  MIT 
RESTRICTION 


Message 

Category 

Baseline 

Atlanta 

Data  Link 
Team  1 

Data  Link 
Team  2 

Data  Link 
Team  3 

Speed 

7 

0 

0 

0 

Heading 

7 

1 

2 

3 

Altitude 

56 

7 

7 

6 

Combination 

Clearance 

6 

1 

0 

1 

TOC 

40 

4 

4 

4 

Route 

9 

0 

3 

5 

Crossing 

Restriction 

1 

0 

0 

0 

Initial  Contact 
Response 

13 

2 

1 

1 

Information 

7 

0 

0 

3 

Request 

Information 

19 

5 

1 

1 

Correction 

10 

1 

1 

1 

22 

2 

1 

2 
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controllers  on  the  baseline  day  and  during  the  corresponding  Data  Link 
test  runs.  As  shown  in  the  table,  the  availability  of  Data  Link  affected  all 
types  of  controller  voice  messages,  with  major  reductions  accounted  for  by 
altitude  clearances,  transfer  of  communication  messages,  responses  to 
initial  contact  calls,  and  “other”  messages  primarily  acknowledgments  of 
messages  received  from  pilots). 

While  they  could  not  be  replaced  by  the  set  of  Data  Link  messages  used  in 
this  experiment,  it  is  interesting  to  note  that  voice  radio  corrections 
(repeats  and  clarifications)  and  requests  for  information  from  the  pilots 
(e.g.,  “are  you  on  frequency?”,  “say  speed”)  were  reduced  during  Data  Link 
testing.  This  finding  suggests  that  the  persistent,  visual  controller  and  pilot 
displays  associated  with  Data  Link  may  have  improved  the  effectiveness  of 
ATC  communications  by  reducing  the  need  for  redundant  voice 
transmissions. 

5. 1.5.2  Data  Link  Usage. 

The  sector  32  control  teams  sent  an  average  of  109  Data  Link  messages 
during  each  of  the  five  test  runs.  All  uplinks  were  either  control  clearances 
or  TOC  messages.  No  informational  messages,  corrections,  or  requests  for 
information  were  sent  using  Data  Link. 

Table  2  presents  a  breakdown  of  the  Data  Link  messages  sent  by  each  of 
the  control  teams  during  the  20  MIT  test  runs  that  were  directly 
comparable  to  the  historical  baseline  day  under  voice-only 
communications.  As  shown  in  the  table,  manually  entered  altitude 
messages,  predefined  clearances  selected  using  the  menu  text  function  and 
TOC  uplinks  were  the  most  frequently  employed  Data  Link  messages.  For 
this  study,  the  menus  were  customized  by  the  individual  teams  prior  to 
testing  to  include  messages  that  they  expected  to  send  on  a  repetitive 
basis.  In  sector  32,  the  menus  for  all  groups  consisted  of  one  or  two 
altitude  cleeirances,  a  number  of  route  clearances  to  fixes,  and  crossing 
restrictions. 

Table  2  also  indicates  that  the  three  teams  used  Data  Link  to  send  all  TOC 
messages  to  the  36  equipped  aircraft.  While  teams  2  and  3  used  the 
automatic  TOC  mode  exclusively  to  send  these  messages,  team  1  used  the 
manual  mode  for  approximately  one-half  of  the  aircraft. 

Figure  15  summarizes  the  teams’  use  of  Data  Link  across  the  five  test 
conditions.  The  histograms  present  the  number  of  messages  averaged 
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TABLE  2.  DATA  LINK  MESSAGES  IN  SECTOR  32  ON  THE  DATA  LINK 
TEST  DAYS  AT  THE  20  MIT  RESTRICTION 


Message 

Category 

Data  Link 
Team  1 

Data  Link 
Team  2 

Data  Link 
Team  3 

Speed 

11 

9 

8 

Heading 

11 

0 

1 

Altitude 

57 

22 

16 

Combination 

Clearance 

0 

0 

1 

22 

34 

43 

2 

1 

2 

TOC 

Auto 

16 

36 

36 

TOC 

Manual 

20 

0 

0 

(1)  All  items  selected  from  menu  -  includes  additional  altitude  clearances,  route 
clearances  direct  to  a  fix  and  crossing  restrictions. 

(2)  Text  messages  composed  on-line  -  includes  additional  headings,  special 
route  clearances,  and  a  clearance  to  an  altitude  at  a  prescribed  climb  rate. 

across  the  three  teams  and  divided  into  TOC  and  clearance  transmissions. 
While  some  variation  in  the  total  number  of  Data  Link  messages  is  evident 
in  the  graph,  no  consistent  relationship  is  apparent  between  the 
difficulty/complexity  of  the  control  problem  and  the  use  of  the  Data  Link 
system.  Thus,  the  control  teams  did  not  abandon  Data  Link  in  favor  of  the 
traditional  voice  channel  when  traffic  density  increased  or  when  the 
number  of  aircraft  were  raised  by  10  percent. 
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FIGURE  15.  DATA  LINK  MESSAGES  SENT  IN  SECTOR  32 
5.2  EXPERIMENT  2  -  ARRIVAL  SECTOR. 

As  described  in  section  4,  the  hypothesized  user  benefit  of  Data  Link 
communications  for  experiment  2  was  an  improvement  in  the  efficiency 
with  which  aircraft  would  be  controlled  in  sector  09  and  delivered  to  the 
arrival  fix  for  the  Atlanta  airport.  The  following  subsections  first  present 
data  on  the  criterion  safety  measures  that  were  collected  on  each  of  the 
five  test  runs  completed  by  the  three  teams  of  sector  09  controllers.  Sector 
throughput  and  efficiency  data  are  then  presented  for  the  test  runs  that 
were  safely  accomplished  by  the  subjects. 

5.2.1  Criterion  Performance. 

5.2. 1.1  Aircraft  Separation. 

Aircraft  separation  was  continuously  monitored  at  the  Host  operator’s 
console  during  each  test  run.  No  operational  errors  (minimum  en  route 
separation  violations)  were  recorded  between  aircraft  in  sector  09  on  any 
of  the  five  test  runs  for  any  of  the  three  teams  of  controllers. 


20  MIT  15  MIT  10  MIT  5  Miles  5  Miles 

Data  Link  Test  Condition  110%Traf 
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5,2. L2  Operational  Safety  Assessments. 


Controller  and  Supervisor  Ratings 

Expert  operational  assessments  of  the  safety  with  which  the  test  runs  were 
completed  are  shown  in  figure  16.  The  histogram  presents  the  frequency 


Higher  Margin 

Typical  for 

Safe-Some 

Unsafe 

of  Safety 

Sector 

Concern 

Safety  Rating 

FIGURE  16.  SECTOR  09  SAFETY  RATINGS 


with  which  the  15  test  runs  (5  per  team)  received  each  of  the  4  possible 
safety  ratings  from  the  radar,  data  and  tracker  controllers  as  well  as  the 
observing  supervisor  for  the  sector. 

None  of  the  controllers  or  supervisors  rated  any  of  the  test  runs  as 
“unsafe.”  As  in  experiment  1,  the  supervisors  assigned  the  test  runs  the 
highest  safety  ratings  overall  with  seven  runs  rated  as  having  a  greater 
margin  of  safety  than  normal  for  the  sector  in  the  operational,  voice-only 
environment,  and  eight  runs  rated  as  having  typical  levels  of  safety.  The 
controllers  rated  the  margin  of  safety  of  approximately  one-half  of  the  runs 
as  typical  for  the  sector  (24  of  45  ratings),  with  the  remaining  runs  rated  as 
having  a  higher  margin  of  safety  than  normal.  No  ratings  of  “safe,  but 
some  concern”  were  received. 
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Examination  of  the  ratings  across  the  test  runs  revealed  no  evidence  for  a 
reduction  in  perceived  safety  with  changes  in  the  number  of  aircraft  under 
control.  On  the  1  O^gher  margin  of  safety)  to  4  (unsafe)  scale,  average 
team  ratings  for  the  three  groups  of  controllers  and  their  supervisors  were 
consistently  at  or  below  1.67  from  the  first  run  with  baseline  traffic  levels 
to  the  last  run  in  which  the  number  of  aircraft  arriving  at  Atlanta  was 
increased  by  40  percent. 

Pilot  Ratings 

Pilot  safety  assessments  were  performed  as  part  of  a  post  run 
questionnaire  completed  by  the  two  pilots  who  flew  each  full  fidelity  flight 
simulator  during  the  test  runs,  and  by  simulation  site  coordinators  who 
acted  as  observers.  They  rated  the  flights  on  a  scale  ranging  from 
“completely  safe”  (1)  to  “completely  unsafe”  (7). 

The  FAA  RCS  (B747),  NLR  B747,  Avia  727,  and  FAA  GAT  simulators 
participated  in  all  sector  09  flights,  while  the  OKC  B727  participated  in 
only  the  final  two  replications  of  the  study.  All  five  simulators  were 
equipped  to  receive  Data  Link  messages.  On  each  test  run,  the  Avia  and 
RCS  completed  one  test  flight,  while  the  remaining  simulators  completed 
two  test  flights.  This  yielded  a  total  of  110  safety  rating  opportunities  over 
the  entire  experiment. 

For  the  purpose  of  analysis,  the  three  ratings  for  each  flight  were  summed 
to  produce  a  scale  with  a  minimum  value  of  3  (completely  safe)  to  21 
(completely  unsafe).  Of  the  1 10  flights,  75  (68  percent)  received  combined 
ratings  of  3,  indicating  unanimous  agreement  among  the  raters  that  the 
flight  had  been  completely  safe.  Of  the  remaining  35  flights,  33  combined 
ratings  were  below  the  combined  scale  midpoint  of  “12”,  with  22  of  these 
rated  as  “5”  or  lower. 

Two  of  the  110  flights  received  combined  scores  higher  than  the  scale 
midpoint.  Both  of  these  were  produced  in  flights  completed  by  the  same 
simulator.  In  one  case  the  observer,  PF  and  PNF  assigned  the  flight 
individual  ratings  of  5,  5,  and  7,  respectively  on  the  1  to  7  scale.  Written 
comments  from  the  three  raters  indicated  that  the  poor  safety  judgments 
were  caused  by  a  flaw  in  the  operation  of  the  Data  Link  pilot  interface  used 
in  the  simulator.  The  pilots  were  unable  to  use  the  recall  feature  of  the 
equipment  to  review  an  earlier  ATC  message  because  the  displayed  pointer 
was  not  correctly  aligned  with  the  appropriate  actuation  button  on  the 
display  bezel. 
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In  the  second  case,  the  observer,  PF  and  PNF  assigned  the  flight  ratings  of 
6,  7,  and  7,  respectively.  Comments  from  these  raters  implicated  a 
software  failure  in  the  simulation  equipment  or  noise  in  the  long  distance 
data  transmission  system.  At  one  point  during  the  flight,  the  pilots 
reported  a  rapid  influx  of  Data  Link  messages  presented  on  the  “message 
pending”  display.  No  corresponding  uplinks  had  been  sent  by  the  test 
controllers.  When  called  to  the  display,  the  “messages”  were  found  to  be 
data  error  messages.  Each  of  these  disrupted  the  Data  Link  connection 
and  prevented  further  communication  during  the  test  flight. 

The  incidents  described  above  indicate  that  the  two  poor  safety  ratings 
were  attributable  to  local  problems  in  a  single  simulator.  Since  neither 
rating  was  caused  by  controller  error  or  by  inherent  problems  with  Data 
Link  ATC  communications,  the  associated  test  runs  were  not  judged 
invalid. 

5.2.2  Sector  Throughput. 

The  findings  presented  in  section  5.2.1  indicate  that  the  control  teams  in  all 
three  replications  of  the  study  were  able  to  safely  control  the  baseline 
aircraft  sample  in  sector  09,  as  well  as  the  increases  in  Atlanta  arrival 
traffic  ranging  from  10  to  40  percent.  The  following  three  subsections 
present  data  on  changes  in  sector  throughput  and  aircraft  operating 
efficiency  associated  with  the  use  of  Data  Link  to  supplement  the  ATC 
communications  channel  in  this  sector. 

5.2.2. 1  Baseline  Traffic. 

The  central  question  that  was  addressed  in  this  experiment  was  whether 
sector  09  control  teams  using  a  combined  voice  and  Data  Link 
communications  system  could  improve  the  efficiency  with  which  Atlanta 
arrival  traffic  typically  is  controlled  during  the  target  rush  period.  Figure 
17  presents  the  mean  time  and  distance  flown  by  Atlanta  arrival  aircraft  on 
the  historical  baseline  day  and  on  the  three  Data  Link  test  runs  which 
duplicated  the  baseline  day  traffic. 

As  shown  in  the  figure,  the  addition  of  Data  Link  communications  was 
associated  with  a  reduction  in  mean  Atlanta  arrival  aircraft  flight  time  and 
distance  in  sector  09.  The  average  arrival  aircraft  on  the  historical  baseline 
day  flew  111  miles  in  the  sector  over  more  than  18  minutes.  In  contrast, 
when  under  control  of  the  Data  Link  teams,  the  average  aircraft  flew  less 
than  89  miles  over  approximately  14  minutes. 
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-  120 

Time  in  Sector  m  Distance  Flown 


Baseline  Atlanta  Team  1  Team  2  Team  3 

Voice-Only  Data  Link 

FIGURE  17.  AVERAGE  AIRCRAFT  TIME  AND  DISTANCE 
FLOWN  IN  SECTOR  09 

As  in  experiment  1,  traditional  statistical  significance  testing  of  these 
findings  was  not  possible  because  of  the  small  number  of  control  teams 
that  could  be  tested.  However,  the  practical  significance  of  the  findings 
can  be  inferred  from  figure  17  by  noting  the  consistent  improvement  in 
sector  throughput  displayed  by  the  three  independent  test  teams  in 
comparison  to  the  voice-only  control  team  on  the  representative  historical 
sample  day. 

An  alternative  statistical  analysis  comparable  to  that  performed  on  the 
data  from  experiment  1  was  conducted  by  treating  the  individual  Atlanta 
arrival  aircraft  as  subjects,  and  comparing  each  aircraft’s  flight  efficiency 
on  the  baseline  day  to  its  performance  under  the  three  test  teams’  control 
on  the  simulation  run  with  identical  traffic  demands.  The  repeated 
measures  analysis  of  variance  verified  that  arrival  efficiency  with  Data 
Link  was  significantly  better  than  that  on  the  baseline  day  with  voice-only 
communications  (F13 42=5.80,  p<.01). 
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In  agreement  with  the  results  for  experiment  1,  figure  17  also  shows  that 
the  flight  time  and  distance  measures  were  closely  associated,  indicating 
that  the  controllers  did  not  use  potentially  inefficient  speed  increases  to 
decrease  time  in  sector.  Evaluation  of  the  aircraft  track  profiles 
suggests  that  other  control  techniques  were  employed  during  Data  Link 
testing  to  increase  flight  efficiency. 

Figure  18  presents  the  horizontal  track  plots  for  a  sample  of  15  Delta 
Airlines  aircraft  that  flew  during  the  historical  rush  period  and  during  Data 
Link  test  scenarios  which  duplicated  the  baseline  traffic  pattern.  Each  of 
the  plots  shows  the  aircraft  sample  entering  sector  09  and  proceeding  into 
the  sequence  to  cross  the  Tiroe  arrival  fix  at  the  specified  altitude 
restriction.  On  the  historical  baseline  day,  it  can  be  seen  that  the  sequence 
was  merged  at  a  greater  distance  from  Tiroe  than  on  the  Data  Link  test 
trials.  Sequencing  at  the  merging  point  on  the  baseline  day  was 
accomplished  in  a  tactical  fashion,  with  most  aircraft  vectored  into  a 
holding  pattern  to  attain  their  positions  in  the  arrival  sequence. 

In  contrast,  the  controllers  using  the  combined  voice  and  Data  Link 
communications  system  were  able  to  provide  more  strategic  ATC  service 
which  resulted  in  much  more  efficient  aircraft  operation.  While  some 
vectoring  to  achieve  sequencing  is  apparent,  none  of  the  aircraft  were 
required  to  enter  holding  patterns.  In  addition,  many  of  the  aircraft  appear 
to  have  been  cleared  directly  to  Tiroe,  providing  them  with  “short  cut” 
routes  to  the  arrival  fix. 

Figure  19  separates  the  flight  time  and  distance  data  for  the  three 
categories  of  air  traffic  that  flew  in  sector  09  on  the  historical  baseline  day 
and  during  Data  Link  testing  with  the  identical  traffic  pattern.  As  shown  in 
the  figure,  the  23  Atlanta  arrivals  reflect  the  improvement  in  flight 
efficiency  that  was  discussed  above.  The  other  traffic  in  the  scenario  were 
4  aircraft  bound  for  the  satellite  Peachtree  Dekalb  airport  (PDK),  and  14 
overflights.  The  testing  results  show  no  appreciable  change  from  the 
baseline  day  in  flight  time  or  distance  for  the  PDK  arrivals,  and  an 
apparent  increase  in  efficiency  for  the  overflights.  This  finding  confirms 
that  the  improvements  for  aircraft  in  the  primary  Atlanta  arrival  traffic 
flow  were  not  attained  by  sacrificing  ATC  service  to  other  aircraft. 

5.2.2.2  Increased  Traffic. 

The  final  four  simulation  runs  completed  under  Data  Link  testing  were 
used  to  assess  the  control  teams’  ability  to  deal  with  increases  in  the 
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FIGURE  19.  TIME  AND  DISTANCE  FLOWN  IN  SECTOR 
FOR  ATLANTA  ARRIVALS,  PDK  ARRIVALS, 

AND  OVERFLIGHTS 

Atlanta  arrival  traffic  sample.  A  fast-time  SIMMOD  model  was  used  to 
provide  comparative  baseline  data  for  flight  efficiency  data  collected 
during  these  runs. 

The  detailed  SIMMOD  model  of  the  sector  09  traffic  flow  was  developed 
by  analyzing  traffic  routing,  resulting  aircraft  interactions,  and  travel 
times  from  the  historical  baseline  sample  day  rush  period.  The  model 
parameters  were  adjusted  to  duplicate  the  actual  traffic  flow  by  imposing 
the  varied  MIT  restrictions  for  aircraft  at  the  Tiroe  fix  that  were  used  on 
the  sample  day,  and  by  eliminating  interactions  between  the  jet  and 
turboprop  aircraft  that  flew  at  different  altitudes.  The  adjusted  model 
produced  a  close  match  to  the  actual  baseline  data  derived  from  the 
historical  SAR  tape.  The  model  calculations  yielded  an  average  time  in 
sector  estimate  of  18.12  minutes  in  comparison  to  18.32  minutes  for  the 
actual  baseline  day. 


Baseline  data  for  comparison  to  the  Data  Link  test  results  were  generated 
by  challenging  the  capabilities  of  the  voice  communications-based 
SIMMOD  sector  model  with  increased  traffic  levels.  The  timing  and 
sector  entry  points  of  the  flights  that  were  added  to  the  input  file  to  the 
model  were  identical  to  those  added  to  the  scenarios  for  the  110,  120,  130, 
and  140  percent  test  runs. 

Figure  20  compares  the  modeled  baseline  data  to  that  obtained  under 
testing  with  Data  Link.  Increasing  the  number  of  arrival  aircraft  from  25  to 
31  produced  an  expected  average  increase  in  the  total  aircraft  time  in 
sector  both  in  the  voice  communications-based  model  and  during  Data 
Link  testing.  As  shown  in  the  figure,  however,  the  overall  reduction  in 
average  aircraft  sector  time  demonstrated  during  Data  Link  testing  with 
the  baseline  number  of  aircraft  was  maintained  as  traffic  levels  increased. 
In  comparison  to  the  voice  communications-based  model  estimates.  Data 
Link  saved  from  3  to  4  minutes  in  sector  flight  time  for  the  average  aircraft 
when  traffic  levels  rose  by  10  to  40  percent. 
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FIGURE  20.  AVERAGE  TIME  IN  SECTOR  09  FOR  INCREASED  TRAFFIC 
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In  addition,  examination  of  average  sector  flight  times  indicates  that  the 
disparity  between  Data  Link  and  voice-only  communications  may  increase 
as  traffic  demand  is  raised.  While  the  average  reduction  in  sector  flight 
time  with  Data  Link  was  3.07  minutes  at  the  1 10  percent  level  of  traffic,  the 
savings  increased  to  3.4  minutes  at  120  percent,  3.83  minutes  at  130 
percent,  and  3.96  minutes  at  140  percent.  Thus,  it  appears  that  the  overall 
increase  in  the  capacity  of  the  communications  channel  with  Data  Link 
may  reduce  the  rate  of  growth  in  average  aircraft  flight  time  as  the  system 
is  challenged  by  increasing  traffic  demands. 

Figure  21  shows  the  processing  time  for  all  Atlanta  arrival  aircraft 
averaged  across  teams  for  each  test  condition.  Processing  time  was 
calculated  for  the  baseline  day  and  the  test  conditions  by  subtracting  the 
problem  exit  time  for  the  first  Atlanta  arrival  aircraft  leaving  sector  09 
from  that  of  the  last  aircraft.  The  problem  exit  time  for  each  aircraft  was 
defined  as  the  time  of  day  at  which  the  aircraft  crossed  the  Tiroe  arrival 
fix.  Thus,  processing  time  reflects  the  time  needed  to  complete  the  sector’s 
task  of  merging  the  traffic  sample  during  the  rush  period  in  order  to  place 
the  aircraft  on  their  common  arrival  route  into  the  low  altitude  terminal 
area. 

1st  Aircraft  Exit  Time  -  Last  (Minutes) 
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FIGURE  21.  ARRIVAL  TRAFFIC  PROCESSING  TIME  IN  SECTOR  09 
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This  view  of  the  data  demonstrates  that  the  reduced  flight  times  and 
distances  discussed  above  were  effectively  realized  in  an  overall 
improvement  in  the  productivity  of  the  sector.  As  shown  in  the  figure,  over 
50  minutes  were  needed  to  process  all  of  the  arrivals  on  the  historical 
baseline  day.  Under  testing  where  the  communications  channel  was 
expanded  by  Data  Link,  processing  time  was  reduced  by  an  average  of 
14.8  minutes  (29  percent)  for  the  identical  traffic  sample.  The  data  also 
show  that  Atlanta  arrival  processing  time  was  reduced  even  when  the 
arrival  traffic  sample  was  increased  by  up  to  eight  aircraft. 

Section  6  presents  estimates  of  the  economic  benefits  associated  with  the 
in-flight  efficiency  improvements  demonstrated  in  this  experiment. 

5.2.3  Sample  Fuel  Consumption. 

As  discussed  in  section  4.7,  this  experiment  focused  on  measurements  of 
flight  time  and  distance  to  assess  aircraft  operating  efficiency.  However, 
fuel  consumption  data  were  recorded  for  piloted  aircraft  simulators  in 
order  to  provide  representative  examples  of  the  way  in  which  these  savings 
would  be  reflected  in  reduced  usage  of  resources.  Estimates  of  fuel 
consumption  were  calculated  by  models  associated  with  four  of  the 
simulators  that  participated  as  Atlanta  arrival  flights.  All  comparisons 
were  made  between  the  historical  baseline  flights  and  the  initial  test  runs 
with  Data  Link  in  which  the  traffic  sample  duplicated  that  on  the  baseline 
day. 

Table  3  presents  the  average  reduction  in  flight  time  and  corresponding 
fuel  savings  for  four  flights  completed  by  the  simulators  during  testing. 

All  four  of  these  flights  had  received  extensive  vectoring  or  had  been 
directed  to  a  holding  pattern  on  the  historical  sample  day.  In  all  cases 
under  Data  Link  testing,  these  flights  achieved  an  earlier  arrival  at  the 
Tiroe  fix  and  consumed  less  fuel. 

5.2.4  Secondary  Criterion  Measures. 

The  primary  operational  safety  measures  described  in  section  4  were  used 
as  criteria  for  validating  the  NAS  user  benefits  identified  during  testing. 

As  in  experiment  1,  a  secondary  group  of  measures  was  used  in  this 
experiment  to  determine  whether  the  user  benefits  were  achieved  at  any 
expense  to  the  general  quality  of  task  performance  exhibited  by  the 
controllers  or  any  significant  increase  in  the  workload  levels  that  they 
experienced. 
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TABLE  3.  SAMPLE  AVERAGE  FLIGHT  TIME  AND  FUEL  SAVINGS  FOR 
SELECTED  SECTOR  09  FLIGHTS  FLOWN  BY  FLIGHT 
SIMULATORS 


Atlanta  Arrival  Flight 
(Simulator) 

Flight  Time  Savings 
(minutes) 

Fuel  Savings 
(pounds) 

DAL  772  (NLRB747) 

18.18 

3827 

DAL  16  (RCS  B747) 

10.07 

1811 

DAL  952  (AviaB727) 

10.29 

884 

DAL  165  (OKCB727) 

14.09 

572 

5.2.4.1  Supervisor  Performance  Ratings. 

During  each  test  run,  the  sector  09  supervisors  judged  the  control  team  on 
nine  factors  indicative  of  degraded  sector  performance.  A  5-point  rating 
scale  ranging  from  1  “never  occurred”  to  5  “occurred  unacceptably  often” 
was  used  to  quantify  the  judgments. 

Ratings  on  each  dimension  for  the  five  test  conditions  averaged  across  the 
three  supervisors  are  summarized  in  figure  22.  None  of  the  average 
ratings  exceeded  the  midpoint  on  the  scale  (3  -  “within  normal  limits  for 
the  sector  during  this  duty  period”).  Inspection  of  the  data  for  individual 
teams  indicated  that  all  of  the  five  ratings  of  the  nine  factors  made  by  each 
supervisor  were  “within  normal  limits”  or  better.  As  shown  in  figure  22, 
the  mean  ratings  also  did  not  reveal  any  tendency  for  deterioration  in 
sector  performance  as  the  traffic  demands  were  increased  across  the  test 
runs. 

5.2.4.2  Controller  Workload. 

Controller  workload  was  assessed  using  a  subjective  rating  scale  of 
perceived  effort.  Following  each  test  run,  the  controllers  rated  their 
workload  on  a  5-point  scale  in  comparison  to  their  normal  workload  at 
sector  09  for  the  same  work  period.  Figure  23  presents  the  mean 
workload  ratings  for  each  control  position  across  the  five  test  runs. 

As  indicated  by  figure  23,  the  mean  ratings  suggest  that  there  was  no 
apparent  trend  for  average  workload  ratings  to  increase  when  the  traffic 
demand  was  raised  over  the  five  test  runs.  However,  the  data  do  reveal  a 
consistent  difference  in  relative  workload  across  the  three  control 
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FIGURE  23.  SECTOR  09  CONTROLLER  WORKLOAD  RATINGS 
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positions.  Regardless  of  the  test  condition,  all  three  controllers  at  the  radar 
position  indicated  that  their  workload  was  either  “about  the  same”  or  lower 
than  that  for  their  position  at  the  operational  sector  09  during  the  same 
rush  period.  In  contrast,  the  ratings  received  from  the  controllers  at  the 
data  and  tracker  positions  sometimes  suggested  that  workload  was  either 
“somewhat”  or  “much  higher.” 

Specifically,  the  tracker  controllers  on  two  teams  consistently  rated  their 
workload  as  somewhat  higher,  while  the  third  indicated  that  it  was  about 
the  same.  Likewise,  the  data  controller  on  one  of  the  teams  indicated  that 
his  workload  was  “much  higher  than  normal,”  while  the  remaining  two 
consistently  rated  their  workload  as  about  the  same  or  somewhat  lower. 

Rather  than  indicating  a  potential  workload  problem  during  testing,  these 
ratings  are  likely  to  be  indicative  of  the  redistribution  of  sector  tasking  that 
occurred  when  the  combined  Data  Link  and  voice  radio  communications 
capability  was  introduced.  As  discussed  in  a  later  section  of  these  results, 
the  control  teams  in  both  experiments  adopted  procedures  under  which 
the  radar  controller  performed  all  voice  radio  communications  and 
directed  the  tracker  and  data  controllers  to  send  and  monitor  Data  Link 
transactions  (see  section  5.3.1).  Among  other  effects,  this  freed  the  radar 
controller  to  coordinate  and  share  control  decision  making  with  his 
associates.  Such  team  activity  contrasted  sharply  with  a  typical  busy 
sector  under  voice-only  communications  where  the  radar  position  can  be 
completely  occupied  with  communications  and  the  associates’  assistance  is 
necessarily  confined  to  peripheral  duties. 

This  explanation  of  the  somewhat  disparate  workload  ratings  was 
supported  by  written  comments  solicited  after  each  test  run.  The 
controller  who  had  rated  all  runs  as  having  much  higher  workload  than 
normal  at  sector  09  during  the  same  work  period  indicated  that  his 
workload  at  the  data  position  was  higher  because  he  was  able  to  help 
much  more  than  he  normally  could.  Rather  than  being  detrimental,  the 
workload  increase  was  associated  with  increased  productivity. 

Similar  comments  were  obtained  from  controllers  who  staffed  the  tracker 
position  to  explain  their  relative  workload  ratings.  None  of  the  ratings 
obtained  from  any  of  the  teams  were  accompanied  by  comments 
suggesting  that  increases  in  controller  workload  had  posed  any  risk  of 
poorer  performance. 
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5.2 .4.3  Handoff  Measures. 


Measures  of  controller  performance  in  accepting  handoffs  from,  and 
offering  handoffs  to,  adjacent  sectors  were  collected  during  the  experiment 
as  subsidiary  workload  indices.  Figure  24  presents  results  for  two  handoff 
measures  obtained  from  the  baseline  Atlanta  SAR  tape  and  the  comparable 
baseline  traffic  test  run  with  Data  Link.  As  discussed  below,  neither 
measure  indicated  that  controller  workload  under  Data  Link  testing  had 
affected  sector  handoff  performance. 

The  0-A  Interval  is  an  index  of  the  average  time  taken  by  the  control  team 
to  accept  an  aircraft  handoff  from  a  sending  sector.  As  discussed  in 
experiment  1,  the  use  of  this  measure  was  based  on  the  hypothesis  that 
unacceptable  increases  in  workload  may  have  been  reflected  by  delays  in 
accepting  the  handoff  offer.  No  evidence  of  controller  overload  during 
Data  Link  testing  was  provided  by  this  measure.  In  fact,  as  shown  in  figure 
24,  handoffs  were  accepted  an  average  35  seconds  earlier  during  testing 
than  they  were  on  the  baseline  day. 

The  O-E  Distance  Flown  is  an  index  of  the  average  distance  from  the 


Atlanta  Baseline  Data  Link  Test 

■Offer-Accept  interval  M  Offer-Exit  distance  to  boundry 


FIGURE  24.  SECTOR  09  HANDOFF  MEASURES 
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boundary  at  which  the  sector  09  controllers  offered  a  handoff  to  the  next 
sector  along  the  aircraft’s  route  of  flight.  It  was  hypothesized  that 
excessive  workload  in  sector  09  may  have  been  evidenced  either  by 
offering  handoffs  earlier  to  shed  load,  or  by  an  inappropriate  delay  in  the 
handoff  caused  by  overload  attributable  to  other  sector  tasks. 

The  results  show  that,  under  Data  Link  testing,  the  controllers  offered 
handoffs  an  average  of  5.1  miles  closer  to  the  sector  boundary  than  their 
counterparts  did  on  the  historical  baseline  day.  The  mean  flight  distance 
and  time  to  the  exit  boundary  were  approximately  25.4  miles  and  5.1 
minutes,  in  comparison  to  30.5  miles  and  5.8  minutes  for  the  baseline  day. 
Both  sets  of  values  are  within  normal  limits  for  offering  handoffs. 

As  in  sector  32,  the  relatively  reduced  flight  distance  to  the  sector 
boundary  following  a  handoff  offer  that  was  observed  during  sector  09 
Data  Link  testing  appears  to  have  been  the  result  of  a  strategy  difference 
between  normal  voice  and  Data  Link  communications  procedures  rather 
than  load  shedding.  In  order  to  make  effective  use  of  the  automatic  Data 
Link  TOC  while  retaining  communications  with  the  aircraft  for  the 
required  period  of  time,  the  controllers  adopted  a  strategy  of  delaying  the 
handoff  offer.  As  shown  in  the  figure,  this  decreased  the  0-E  Distance. 
However,  as  indicated  by  the  “within  normal  limits”  supervisor  ratings  for 
handoff  performance  (section  5.2.4. 1)  and  by  the  acceptable  flight  time  and 
distance  to  the  boundary  that  were  achieved,  the  strategy  had  no 
discernible  negative  impact  on  ATC  system  performance. 

5.2.5  Communications. 

Measures  of  the  nature  and  content  of  communications  conducted  on  the 
baseline  day  and  during  testing  were  obtained  as  secondary  outcome 
measures.  As  discussed  in  section  4.7.2.2,  it  was  expected  that  any  user 
benefit  observed  during  testing  would  be  associated  with  a  redistribution 
of  communications  between  the  voice  and  Data  Link  media. 

5.2.5. 1  Voice  Radio  Usage. 

Figure  25  illustrates  the  reduction  in  voice  radio  usage  that  occurred 
during  testing  when  Data  Link  was  available  for  ATC  communications.  As 
shown  in  the  figure,  during  the  rush  period  on  the  baseline  day  at  the 
Atlanta  ARTCC,  the  radio  channel  was  occupied  for  more  than  16  minutes. 
During  Data  Link  testing,  radio  channel  occupation  time  for  the  average 
sector  09  control  team  during  all  five  test  runs  dropped  by  78  percent  to  3.5 
minutes. 


67 


Minutes  of  Channel  Occupation 


Baseline  Atlanta 

Team  1 

Team  2 

Team  3 

Voice-Only 

Data  Link  +  Voice 

FIGURE  25.  VOICE  RADIO  CHANNEL  OCCUPATION 
TIME  IN  SECTOR  09 


Detailed  analyses  of  the  number  and  content  of  voice  messages  were 
completed  for  the  historical  baseline  rush  period  in  sector  09  and  the 
comparable  baseline  traffic  Data  Link  tests.  On  the  baseline  day,  the 
controllers  made  159  voice  transmissions  while  pilots  made  183 
transmissions.  During  Data  Link  testing,  the  total  number  of  controller 
voice  transmissions  dropped  to  an  average  of  48.7  while  pilot  transmissions 
fell  to  an  average  of  44.7.  This  equates  to  a  73  percent  reduction  in 
combined  pilot  and  controller  voice  messages. 

Table  4  presents  a  breakdown  of  the  types  of  voice  messages  sent  by 
controllers  on  the  baseline  day  and  during  the  corresponding  Data  Link 
test  runs.  As  shown  in  the  table,  the  availability  of  Data  Link  affected 
almost  all  types  of  controller  voice  messages,  with  major  reductions 
accounted  for  by  altitude  clearances,  crossing  restrictions,  transfer  of 
communication  messages,  and  responses  to  initial  contact  calls. 
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TABLE  4.  VOICE  RADIO  MESSAGES  IN  SECTOR  09  ON  HISTORICAL 
BASELINE  AND  DATA  LINK  TEST  DAYS  WITH  BASELINE 
TRAFFIC 


Message 

Category 

Baseline 

Atlanta 

Data  Link 
Team  1 

Data  Link 
Team  2 

Data  Link 
Team  3 

Speed 

5 

7 

3 

0 

Heading 

4 

12 

7 

7 

Altitude 

31 

17 

6 

2 

Combination 

Clearance 

5 

0 

2 

1 

TOC 

34 

3 

3 

3 

Route 

10 

1 

3 

6 

Crossing 

Restriction 

25 

3 

2 

5 

Initial  Contact 
Response 

22 

2 

2 

2 

Information 

9 

0 

0 

0 

Request 

Information 

1 

13 

2 

5 

Correction 

11 

4 

1 

4 

Other 

Messages 

9 

10 

3 

5 

52.5.2  Data  Link  Usage. 

The  sector  09  control  teams  sent  an  average  of  143  Data  Link  messages 
during  each  of  the  five  test  runs.  All  uplinks  were  either  control  clearances 
or  TOC  messages.  No  informational  messages,  corrections,  or  requests  for 
information  were  sent  using  Data  Link. 


69 


Table  5  presents  a  breakdown  of  the  Data  Link  messages  sent  by  each  of 
the  control  teams  during  the  test  run  that  was  directly  comparable  to  the 
historical  baseline  day  imder  voice-only  communications.  As  shown  in  the 
table,  manually  entered  altitude  messages,  predefined  clearances  selected 
using  the  menu  text  function  and  TOC  uplinks  were  the  most  frequently 
employed  Data  Link  messages. 

For  both  experiments  of  this  study,  the  menus  were  customized  by  the 
individual  teams  prior  to  testing  to  include  messages  that  they  expected  to 
send  on  a  repetitive  basis.  In  sector  09,  the  menus  for  all  groups  consisted 
of  two  speed  controls  and  a  number  of  route  clearances  to  fixes,  most  with 

TABLE  5.  DATA  LINK  MESSAGES  IN  SECTOR  09  ON  THE 
DATA  LINK  TEST  DAYS  WITH  BASELINE 
TRAFnC 


Message 

Category 

Data  Link 
Team  1 

Data  Link 
Team  2 

Data  Link 
Team  3 

Speed 

15 

14 

5 

Heading 

7 

6 

4 

Altitude 

32 

41 

17 

Combination 

Clearance 

0 

0 

2 

28 

29 

32 

6 

1 

0 

TOC 

Auto 

31 

31 

31 

0 

0 

0 

(1)  All  items  selected  from  menu  -  includes  additional  altitude  clearances,  route 
clearances  direct  to  a  fix  and  crossing  restrictions. 

(2)  Text  messages  composed  on-line  -  includes  additional  headings,  special 
route  clearances,  and  a  clearance  to  an  altitude  at  a  prescribed  climb  rate. 
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crossing  restrictions.  Table  5  also  indicates  that  the  three  teams  used 
Data  Link  to  send  all  TOC  messages  to  the  31  equipped  aircraft.  The 
automatic  TOC  mode  was  used  exclusively  to  send  these  messages. 

Figure  26  summarizes  the  teams’  use  of  Data  Link  across  the  five  test 
conditions.  The  histograms  present  the  number  of  messages  divided  into 
TOC  and  clearance  transmissions  and  averaged  across  the  three  teams.  As 
shown  in  the  figure,  the  mean  number  of  Data  Link  transmissions 
increased  in  a  linear  fashion  with  increases  in  the  size  of  the  traffic  sample 
in  sector  09.  Thus,  the  control  teams  did  not  abandon  Data  Link  in  favor  of 
the  traditional  voice  channel  as  the  complexity  of  the  ATC  scenario  was 
raised  during  testing. 


Baseline  Traffic  110%  Traffic  120%  Traffic  130%  Traffic  140%  Traffic 

Data  Link  Test  Condition 

FIGURE  26.  DATA  LINK  MESSAGES  IN  SECTOR  09 
5.3  SUBSIDIARY  MEASURES. 

The  primary  goal  of  this  study  was  to  identify  and  measure  some  of 
the  user  benefits  associated  with  two-way  Data  Link  ATC  services. 
However,  because  of  the  unique  research  opportunity  provided  by  the 
large-scale,  high  fidelity  simulation  environment,  additional  data  were 
collected  during  both  experiments  to  examine  a  group  of  human  factors 
and  technical  issues  associated  with  Data  Link  ATC  communications. 
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5.3.1  Control  Team  Duty  Profiles. 


As  noted  earlier  in  this  report,  teams  of  three  controllers  were  used  in  both 
experiments  to  duplicate  sector  staffing  during  the  historical  baseline  day 
rush  periods.  Since  this  study  was  the  first  in  which  Data  Link 
communications  were  tested  at  a  sector  manned  by  control  teams  rather 
than  single  radar  controllers,  data  were  collected  to  identify  the  procedures 
adopted  by  the  teams  to  accomplish  communications  and  other  ATC  tasks. 

The  duty  profiles  completed  by  the  controllers  following  each  test  run 
provided  information  on  how  the  control  teams  allocated  tasks  among  the 
three  members.  Correlational  analysis  of  the  data  was  used  to  determine 
whether  the  duty  profiles  provided  by  the  subjects  varied  across  the  five 
test  runs.  The  Kendall’s  Coefficient  of  Concordance  (W)  computed  for  all 
positions  on  each  sector  were  consistently  greater  than  .90,  indicating  that 
each  team  maintained  its  profile  for  division  of  duties  as  testing 
progressed. 

In  addition,  the  data  indicate  that  the  three  teams  of  controllers  who 
participated  at  each  sector  adopted  similar  duty  profiles.  For  sector  32,  the 
duty  profiles  for  the  radar,  data  and  tracker  positions  were  highly 
correlated  across  the  three  teams  (W=  .95,  .92,  and  .98,  respectively). 
Similarly  significant  measures  of  association  were  obtained  in  sector  09 
where  agreement  among  the  teams  on  the  radar,  data  and  tracker  duties 
yielded  W  values  of  .88,  .89,  and  .84,  respectively. 

Because  of  the  high  level  of  agreement  within  sectors  across  test  runs  and 
test  groups,  detailed  analyses  of  the  sector  position  duty  profiles  were 
performed  on  average  scores.  These  scores  were  derived  by  transforming 
ratings  on  the  5-point  scale  ranging  from  “always  my  duty”  to  “never  my 
duty”  to  numeric  values  of  1  to  5,  respectively.  Average  profiles  for  each 
position  at  each  of  the  two  sectors  are  presented  in  tables  6  and  7. 

5.3. 1.1  Key  Duties. 

Examination  of  the  average  profiles  shows  that  both  sectors  were  similar 
in  the  basic  duties  and  responsibilities  assumed  by  the  individual  control 
positions.  In  all  cases,  the  control  teams  departed  from  the  traditional 
standard  of  a  single  controller  communicating  with  aircraft  at  a  sector. 

The  teams  used  the  two  communication  channels  made  available  with  the 
addition  of  Data  Link  by  distributing  communications  tasks  among  all 
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TABLE  6.  SECTOR  32  CONTROL  POSITION  DUTY  PROFILE 
(EXPERIMENT  1) 


Duty  Profile:  Avg.  Rating  1=  Always  2=  Mostly  3=  Occasionally  4=  Rarely  5=Never 

RADAR  DATA  TRACKER 


ALWAYS  1.0 

1.1  Snt.  Voice  Clmc. 

1.2 

1.3  Mon.  Conflict 

1.4  Cntrl.  Decision 

1.5  Mon.  Conform. 

1.6 

1.7  Direct  Team 

1.8 

1.9  Coord,  w/ Team /Voice  TOC 
MOSTLY2.0 

2.1 

2.2 

2.3 

2.4 

2.5 

2.6  Mon.  DL  Trans. 

2.7 

2.8 

2.9  F.S.  Mark 
OCCAS.  3.0 

3.1  DLTOC 

3.2 

3.3 

3.4 

3.5  O/AHandoff 

3.6 

3.7 

3.8 

3.9  Snt.  DL  Clmc. 

RARELY4.0 

4.1 

4.2 

4.3  Land  Comm. 

4.4  Housekeep 

4.5 

4.6 

4.7 

4.8 

4.9 

NEVER  5.0 


Coord  w/Team 


Land  Comm 

Housekeep 
F.S.  Mark 
Mon.  DL  Trans. 


Mon.  Conflict 

Mon.  Conform. 
Cntrl.  Decision 


O/A  Handoff 
DLClmc. 


DLTOC 


Direct  Team 


Voice  Clrnc./ Voice  TOC 


Mon.Conform. 

Mon.  Conflict 

Mon.  DL  Trans/  Housekeep 


DLClmc./ DLTOC/ 

O/A  Handoff/  Coord. w/Team 


Cntrl.  Decision/  Direct 
Team/  Land  Comm. 


Voice  TOC/ Voice  Clrnc./ 
F.S.  Mark 


three  controllers.  In  both  sectors,  the  radar  controller  performed  all  voice 
communications,  while  the  tracker  and  data  controllers  sent  Data  Link 
messages  using  the  radar  and  data  position  keyboards. 

Distributing  the  14  duties  from  the  profile  to  the  control  position  that 
performed  each  most  often,  the  radar  controllers  from  sector  32  typically 
issued  voice  clearances,  monitored  for  aircraft  conflicts,  made  control 
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TABLE  7.  SECTOR  09  CONTROL  POSITION  DUTY  PROFILE 
(EXPERIMENT  2) 


Duty  Profile:  Avg.  Rating  1=  Always  2=  Mostly  3=  Occasionally  4=  Rarely  5= Never 

RADAR  DATA  TRACKER 


ALWAYS  1.0  Voice  Clmc.Woice  TOC 

1.1 

Housekeep 

1.2 

1.3 

Land  Comm. 

1.4  Mon.  Conflict 

1.5  Directed  Team 

1.6 

1.7  Coord  w/Team/  Mon.  Conform. 

1.8 

1.9 

Mon.  Conflict 

MOSTLY2.0  Contrl.  Decision 

Mon.  Conform. 

2.1 

F.S.  Mark/  Coordw/Team 

2.2 

2.3  Mon.  DL  Trans. 

2.4 

2.5 

Mon.  DL  Trans. 

2.6 

2.7  F.S.  Mark 

O/A  Handoff 

2.8 

2.9 

OCCAS.  3.0 

3.1 

DL  Clmc. 

3.2 

3.3 

Contrl.  Decision 

3.4 

3.5  Housekeep 

3.6 

3.7 

Directed  Team 

3.8 

3.9 

RARELY4.0 

4.1  O/A  Handoff 

4.2 

4.3 

DLTOC 

4.4 

4.5  DLTOC 

4.6 

4.7 

4.8  DL  Clmc. 

4.9 

NEVER  5.0  Land  Comm. 

Voice  Clmc./ Voice  TOC 

Mon.  Conform./ Mon.  DL  Trans 
Mon.  Conflict/  Housekeep 
Coord  w/  Team 

DL  TOC/  DL  Clmc./  0/A  Handoff 


Cntrl.  Decision 

Directed  Team/  Land  Comm 


F.S.  Mark 

Voice  TOC/ Voice  Clmc. 


decisions,  directed  other  team  members,  and  sent  voice  TOC  messages. 
The  data  controller  coordinated  actions  with  other  team  members, 
conducted  ground  communications,  performed  sector  housekeeping  and 
marked  flight  strips.  The  tracker  monitored  for  aircraft  conformance,  sent 
Data  Link  clearances  and  TOCs,  monitored  Data  Link  transactions  and 
offered/accepted  sector  handoffs. 
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In  sector  09,  the  radar  controllers  reported  the  most  frequent  performance 
of  the  same  duties  reported  by  sector  32  radar  controllers  with  the  addition 
of  coordinating  their  actions  with  other  team  members.  The  data 
controllers  performed  housekeeping,  ground  communications  and  flight 
strip  marking  most  often,  while  the  trackers’  most  common  tasks  were 
identical  to  their  counterparts  at  sector  32. 

In  general,  these  data  confirm  observations  made  during  testing  that  the 
radar  controllers  acted  as  team  leaders,  insuring  aircraft  separation, 
making  control  decisions,  using  the  radio  to  send  voice  messages,  and 
directing  associates  to  send  Data  Link  messages  to  the  aircraft.  In  both 
sectors,  the  bulk  of  the  Data  Link  inputs  were  made  by  the  tracker  using 
the  radar  position  keyboard  £md  trackball. 

5.3. 1.2  Shared  Duties. 

The  average  ratings  also  indicated  that  a  majority  of  duties  were  shared  by 
two  or  more  controllers.  Other  than  carrying  out  voice  radio 
communications,  and  marking  flight  strips  or  conducting  ground 
communications  in  some  cases,  all  tasks  were  performed  by  more  than  one 
controller.  In  both  sectors,  all  three  controllers  noted  that  they  shared  the 
tasks  of  coordinating  their  actions  within  the  team,  monitoring  for 
conflicts,  monitoring  aircraft  conformance,  and  monitoring  the  progress 
and  outcome  of  Data  Link  transactions  “most  of  the  time.” 

In  addition,  two  controllers  often  shared  other  duties.  For  example,  while 
the  data  controller  reported  marking  flight  strips  more  commonly,  the 
radar  position  often  shared  this  duty  for  clearances  directed  to  the  tracker 
for  uplink.  Likewise,  while  the  tracker  offered  and  accepted  handoffs  most 
often,  the  data  controller  typically  shared  this  task.  Although  more 
common  in  sector  09,  the  data  controller  also  occasionally  sent  Data  Link 
clearances,  and  the  tracker,  followed  by  the  data  controller,  reported 
sharing  the  tasks  of  making  control  decisions  and  directing  the  team  in 
conjunction  with  the  radar  controller. 

5.3. 1.3  Teamwork. 

The  overall  picture  conveyed  by  the  duty  profiles,  and  confirmed  during 
debriefing  discussions,  was  that  the  addition  of  Data  Link  both  demanded 
and  promoted  close  team  cooperation  and  sharing  of  duties.  In  order  to 
make  optimal  use  of  the  expanded  communications  channel,  all  three 
controllers  shared  communications  tasks.  This,  in  turn,  required  extensive 
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intercoordination  among  the  team  members,  and  promoted  a  team 
approach  to  monitoring  the  air  traffic  situation  and  decision  making. 

Unlike  the  current  voice-only  ATC  environment  where  congested  voice 
communications  place  the  data  and  tracker  controllers  in  a  relatively 
passive  position,  the  use  of  Data  Link  made  it  possible  for  the  team  to 
discuss  planned  actions  and  to  inform  one  another  about  actions  that  were 
taken. 

The  duty  profiles  indicate  that  the  precise  nature  of  the  team  interaction 
differed  in  the  two  sectors.  In  sector  32,  all  three  teams  that  participated  in 
the  study  adopted  fairly  fixed  roles  for  each  position  to  control  the 
departure  traffic.  The  tracker  performed  almost  all  Data  Link 
communications  with  occasional  assistance  from  the  data  controller  at  the 
request  of  the  radar  position. 

In  contrast,  sector  09  used  a  more  flexible  strategy  to  sequence  the  arriving 
traffic.  The  radar  controller  assigned  clearances  to  whichever  associate 
was  free  to  make  the  entries,  or  called  out  the  instruction,  and  the  available 
controller  would  make  the  entries. 

As  discussed  above,  both  groups  shared  the  tasks  of  monitoring 
transactions  and  maintaining  awareness  of  the  traffic  situation  among 
team  members.  This  appears  to  have  lead  to  sharing  in  the  decision 
making  and  direction  of  others  on  the  sector.  While  the  radar  controller 
was  clearly  the  team  leader  in  all  cases,  the  associates  in  sector  09,  and  to  a 
lesser  extent  in  sector  32,  often  informed  the  radar  controller  that  they 
were  about  to  take  independent  actions  based  on  a  shared  situation 
awareness  and  understanding  of  immediate  task  requirements.  In  at  least 
one  group,  the  radar  controller  reported  that  he  was  able  to  delegate 
authority  to  a  large  extent  by  simply  instructing  the  data  controller  to 
“keep  this  group  of  aircraft  separated’  while  he  went  on  to  other  tasks. 

5.3.2  Data  Link  Input  Assessments. 

Following  each  test  trial  in  both  experiments,  the  sector  supervisors 
completed  a  questionnaire  regarding  their  observations  of  the  controllers’ 
Data  Link  input  performance  (see  appendix  A).  The  questionnaire  asked 
for  a  basic  rating  of  the  number  of  input  errors  that  were  observed  (“none”, 
“a  few”,  or  “several”).  If  any  errors  were  noted,  the  supervisors  described 
how  the  input  errors  were  handled  by  the  controllers. 
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The  supervisors  reported  that  they  had  observed  no  input  errors  in  13  of 
the  30  test  runs  completed  by  the  2  sectors.  During  17  of  the  runs,  “a  few” 
errors  were  reported.  In  no  case  did  the  supervisors  indicate  that  they  had 
observed  “several”  input  errors. 

For  five  of  the  test  runs  in  which  errors  had  been  observed,  the  supervisors 
said  that  the  controllers  had  detected  the  error  during  the  input  process 
and  had  made  the  appropriate  correction  before  sending  the  message.  In 
the  remaining  cases,  the  controllers  noticed  the  error  in  a  Data  Link 
message  content/status  display,  cmd  corrected  it  with  a  voice  transmission 
or  a  second  Data  Link  message.  In  no  case,  in  either  sector,  did  the 
supervisors  report  that  the  controllers  detected  an  error  only  after  noticing 
an  unintended  aircraft  maneuver,  or  that  the  error  was  never  detected  by 
the  controllers. 

In  a  final  questionnaire  item,  the  supervisors  made  a  projective  evaluation 
of  whether  ATC  Data  Link  will  present  a  greater  potential  for  undetected 
controller  errors  than  the  existing  voice  communications  system.  None  of 
the  responses  from  the  six  Atlanta  ARTCC  supervisors  suggested  that  this 
would  be  the  case. 

5.3.3  Data  Link  Transaction  Times. 

As  discussed  in  the  description  of  the  experimental  methodology  used  in 
this  study,  time  delays  associated  with  uplinking  a  message  to  an  aircraft 
and  downlinking  responses  were  based  on  the  characteristics  of  a 
simulated  Mode  S  transmission  medium.  In  order  to  determine  the  precise 
range  of  transaction  delay  under  which  the  findings  of  this  study  were 
obtained,  time  recordings  were  maintained  for  each  Data  Link  transaction 
completed  in  both  experiments. 

The  measure  of  delay  used  for  analysis  was  Total  Transaction  Time  (TTT). 
TTT  is  defined  as  the  period  of  elapsed  time  from  the  controller’s  input  to 
send  the  message  to  the  appearance  of  a  downlinked  response  on  the 
controller’s  display.  Thus,  the  TTT  includes  the  technical  system  delays 
associated  with  uplink  and  dovmlink,  and  the  time  required  by  the  aircrew 
to  detect,  process,  and  respond  to  the  message.  Since  the  TTT  is  an 
indicator  of  the  overall  delay  experienced  by  the  controller,  it  provides  a 
generic  meems  of  examining  the  impact  of  transaction  duration  on  ATC 
performance. 
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Figure  27  presents  a  frequency  distribution  of  the  I  l  l  s  obtained  during 
this  study.  The  distribution  portrays  the  data  in  2-second  increments  and 
includes  times  for  the  3,699  Data  Link  transactions  that  were  closed  by  a 
“wilco”  or  “unable”  response  from  the  pseudopilots  and  aircraft  simulator 
pilots. 

The  truncation  of  the  distribution  to  the  left  of  the  figure  clearly  reflects 
the  minimum  technical  delays  of  the  transmission  medium,  with  no  TTTs 
below  the  range  of  6.1  to  8  seconds.  Within  the  range  of  6.1  to  20  seconds, 
the  TTTs  appear  to  be  normally  distributed.  The  rapid  drop  in  the 
frequency  of  TTTs  above  20  seconds  identifies  those  cases  where  the  time 
taken  for  the  aircrew  to  respond  delayed  the  downlink  until  the  next 
available  Mode  S  antenna  scan. 

The  full  distribution  of  recorded  TTTs  ranged  from  6.1  to  60  seconds.  The 
mean  TTT  was  15.96  seconds  with  a  standard  deviation  of  6.83  seconds.  Of 
the  total  number  of  transactions,  approximately  83  percent  had  TTTs 
below  20  seconds.  An  additional  13  percent  were  within  the  range  of  20.1 
to  30  seconds.  Only  4  percent  of  the  transactions  took  more  than  30 
seconds  to  complete. 


Total  Tnansaction  Time 


FIGURE  27.  TOTAL  TRANSACTION  TIME  FOR  3699  DATA 
LINK  MESSAGES  SENT  IN  EXPERIMENTS  1 
AND  2 
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5.4  DEBRIEFING. 


Debriefing  sessions  were  conducted  at  the  conclusion  of  each  of  the  three 
replications  of  the  study.  The  controllers  and  supervisors  from  both  test 
sectors  participated  in  combined  groups.  Each  group  was  asked  to 
comment  on  the  quality  of  the  simulation,  the  impact  of  Data  Link  ATC 
communications  in  an  operational  environment,  control  team  procedures, 
and  requirements  for  the  Data  Link  controller  interface.  The  findings  of 
the  three  debriefing  sessions  are  combined  and  organized  by  topic  in  the 
following  subsections. 

5.4.1  Simulation  Fidelity. 

The  initial  question  addressed  by  the  participants  was  whether  they  felt 
that  the  simulation  had  been  sufficiently  realistic,  and  if  the  Data  Link 
system  would  work  as  well  in  the  operational  sectors  as  it  did  during 
testing.  Twenty-three  of  the  24  participants  (18  controllers  and  6 
supervisors)  indicated  that  the  experiments  provided  a  fair  test  of  Data 
Link  effectiveness  in  the  Atlanta  ARTCC  sectors  that  were  simulated.  The 
group  also  agreed  that  the  system  would  work  as  well,  or  better,  in  the 
operational  environment. 

One  of  the  data  controllers  from  sector  32  noted  that,  since  the  baseline 
day  lacked  weather  and  some  of  the  ground  communications  which  can 
complicate  ATC  tasks,  his  workload  during  the  simulation  may  not  have 
been  equivalent  to  that  at  the  test  sector  in  some  instances.  However,  this 
individual  also  indicated  that  he  felt  that  the  Data  Link  services  (especially 
TOC)  would  help  controllers  at  the  operational  sector. 

Other  participants  from  both  sectors  in  two  debriefing  sessions  suggested 
that  the  positive  effects  that  Data  Link  communications  would  have  on 
sector  performance  in  the  real-world  were  underestimated  during  the 
simulation.  These  individuals  noted  that  some  characteristics  of  the 
simulation  made  the  task  of  controlling  the  traffic  more  difficult  than  it 
would  have  been  at  the  operational  sectors.  Included  among  these  were 
the  fact  that  the  simulator  and  pseudopilots  could  not  be  as  responsive  to 
control  instructions  as  pilots  who  tend  to  be  highly  familiar  with  the  ATC 
procedures  in  the  airspace  that  they  traverse  on  a  regular  basis.  In 
addition,  the  controllers  noted  that  surrounding  sectors  in  the  operational 
environment  would  have  provided  more  assistance  by  coordinating  traffic 
and  providing  better  spacing  for  the  aircraft  entering  the  subject  sectors. 
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They  suggested  that  these  factors  made  the  test  a  conservative  evaluation 
of  the  extent  to  which  Data  Link  would  improve  sector  efficiency. 

5.4.2  Data  Link  Effects  on  Controllers  and  on  Service  to  Users. 


Based  on  their  experience  during  simulation  testing,  the  controllers  and 
supervisors  were  asked  to  predict  whether  Data  Link  would  be  an  asset  if 
implemented  today  and,  if  so,  to  describe  its  impact  on  controllers  and  the 
quality  of  service  to  NAS  users.  The  participants  unanimously  agreed  that 
Data  Link  would  improve  the  effectiveness  of  ATC  operations.  The  direct 
effects  on  controllers  that  were  mentioned  included  (1)  a  more  orderly 
work  situation  during  traffic  rushes,  (2)  the  ability  to  transmit  more 
information  in  the  same  period  of  time,  (3)  an  improved  distribution  of 
workload  at  the  sector  and  greater  productivity,  (4)  more  timely  and 
effective  issuance  of  critical  voice  clearances  due  to  reduced  frequency 
congestion,  and  (5)  reduced  communications  errors. 

In  addition,  participants  in  all  three  sessions  noted  that  Data  Link  would 
have  a  profound  effect  on  team  communication  and  effectiveness.  They 
noted  that  during  heavy  traffic  periods  using  only  voice  radio,  the  radar 
controller  may  be  engaging  in  non-stop  communications.  With  Data  Link, 
they  indicated  that  the  control  team  is  able  to  work  together  to  resolve  the 
traffic  problem  and  share  the  workload.  The  data  controller  and  tracker 
are  much  more  aware  of  the  traffic  picture  and  involved  in  the  task  of 
safely  expediting  aircraft  progress.  The  radar  controller  has  the  time  to 
communicate  his  plan,  making  his  associates  more  able  to  share  in  its 
execution  and  detect  when  changes  in  the  plan  are  needed. 

Participants  in  two  of  the  sessions  suggested  that  one  of  the  indirect  effects 
of  Data  Link  implementation  on  the  ATC  system  would  be  a  reduction  in 
sector  staffing  requirements.  These  comments  indicated  that  the  increased 
communications  capacity  provided  by  Data  Link  would  make  it  possible  to 
avoid  the  practice  of  splitting  sectors  since  this  measure  is  often  employed 
when  frequency  congestion  is  excessive.  In  addition,  it  was  noted  that 
adding  a  second  or  third  controller  to  support  a  busy  sector  could  be 
avoided  or  delayed  with  the  availability  of  Data  Link. 

All  24  participants  agreed  that  the  impact  of  Data  Link  on  ATC 
performance  would  enhance  service  to  NAS  users.  Specific  effects 
mentioned  by  the  controllers  that  would  improve  aircraft  operating 
efficiency  included  (1)  the  receipt  of  more  timely  control  clearances 
resulting  in  fewer  maneuvers  and  path  changes,  (2)  reduced  use  of  holding 
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patterns,  (3)  more  efficient  vectoring,  and  (4)  more  fuel  efficient  descent 
and  climb  profiles. 

5.4.3  Data  Link  Transaction  Delays. 

The  debriefing  participants  also  were  asked  to  reflect  on  their  simulation 
experiences  in  order  to  comment  on  any  negative  effects  of  Data  Link 
transaction  delays.  The  subjects  reported  that  delays  were  not  a  problem 
during  testing.  However,  several  suggested  that  transaction  delays  could 
limit  Data  Link’s  utility  if  they  were  excessive.  Participants  in  two 
sessions  indicated  that  the  effects  of  delays  that  they  experienced  were 
outweighed  by  the  increased  ability  to  send  a  voice  message  when  needed 
on  the  clear  radio  cheinnel. 

A  typical  comment  from  controllers  during  each  of  the  debriefing  sessions 
was  the  rapidity  at  which  they  adapted  to  Data  Link  delays  during  training. 
When  asked  to  describe  the  skills  that  were  acquired,  the  controllers  noted 
that  they  quickly  learned  to  judge  when  a  clearance  should  be  sent  by 
voice  rather  than  Data  Link  (e.g.,  a  critical  turn  into  the  arrival  traffic 
sequence).  They  also  indicated  that  they  learned  to  send  Data  Link 
clearances  in  a  more  anticipatory  fashion  to  achieve  the  desired  result. 

5.4.4  Performance  Issues  and  Design  Recommendations. 

The  controllers  identified  two  problems  when  asked  to  identify  potential 
sources  of  error  in  the  use  of  Data  Link.  While  they  agreed  that  typing 
errors  were  possible  with  Data  Link,  the  controllers  indicated  that  such 
errors  would  tend  to  be  corrected  before  the  message  is  sent  when 
observed  in  the  message  composition  area,  or  in  the  Data  Link  status 
displays  after  the  uplink  had  occurred.  However,  it  was  suggested  that  for 
input  errors  to  be  corrected  consistently,  the  controller  making  the  inputs 
would  require  easily  accessible  feedback  on  message  content.  The 
controllers  noted  that  the  composition  area  and  message  status  list  on  the 
PVD  were  not  always  directly  viewable  by  the  data  or  tracker  positions 
located  to  the  left  and  right  of  the  PVD  display. 

A  second  potential  problem  noted  by  the  controllers  was  the  possibility  of 
forgetting  whether  a  message  had  been  sent,  or  the  precise  contents  of  a 
message  after  a  transaction  had  been  closed  and  the  message  content  and 
status  displays  were  cleared.  They  did  not  indicate  that  forgetting  was  any 
more  of  a  problem  with  Data  Link  than  with  the  existing  voice  system. 
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However,  the  controllers  did  agree  that  the  problem  should  be  addressable 
with  Data  Link  by  maintaining  a  recallable  list  of  past  messages. 

Based  on  a  discussion  of  the  issues  outlined  above,  the  controllers  offered 
two  unanimous  recommendations  for  modifications  to  the  design  of  the 
Data  Link  controller  interface.  First,  easily  viewable  feedback  displays 
should  be  provided  for  associate  controllers  engaged  in  entering  and 
monitoring  Data  Link  messages.  Such  displays  should  include  a  feedback 
display  showing  the  message  content  as  it  is  entered,  as  well  as  a  message 
status  list.  In  the  current  Host/PVD  system,  these  might  be  presented  on 
the  data  position  Computer  Readout  Display  (CRD)  unit. 

Second,  a  Data  Link  history  list  should  be  provided.  This  list  should  be 
available  on  command  for  each  aircraft  and  present  the  last  clearance 
message  of  each  type  (i.e.,  altitude,  heading,  and  speed).  The  controllers 
indicated  that  the  history  list  would  be  required  where  control  teams  rather 
than  a  single  controller  staffs  a  sector.  In  these  situations,  the  history  list 
would  resolve  any  question  of  whether  a  message  had  been  sent  by  a  team 
member,  eliminate  unnecessary  discussion,  and  prevent  instances  where  a 
message  is  sent  to  an  aircraft  twice  to  ensure  its  receipt. 

Additional  recommendations  for  system  enhancements  included  an  ability 
to  send  turn  clearances  in  degrees  of  heading  change,  the  capability  to 
save  and  reuse  free  text  messages,  and  a  preloaded  standard  initial 
clearance  that  could  be  sent  to  an  aircraft  entering  the  sector  automatically 
when  it  wilcos  the  TOC  from  the  sending  controller. 

5.4.5  Controller  Team  Procedures  and  Information  Requirements. 

A  final  topic  discussed  during  the  debriefing  sessions  was  Data  Link’s 
implications  for  sector  operations  and  controller  information  requirements. 
There  was  unanimous  agreement  among  the  participants  that  multi¬ 
controller  operation  takes  maximum  advantage  of  the  increased 
communications  capacity  provided  by  Data  Link.  However,  they  also 
noted  that  the  control/communications  team  concept  will  require 
development  of  new  procedural  guidelines  for  sector  operation  and 
accommodation  of  new  information  needs  of  the  team  members. 

Each  control  team  in  this  study  independently  developed  and  tested 
procedures  for  distributing  communications  duties  and  other  sector 
responsibilities  during  training.  As  discussed  in  section  5.3.1,  this  resulted 
in  the  adoption  of  some  operating  rules  that  were  common  across  the 
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teams,  as  well  as  variations  unique  to  individual  teams.  The  successful 
outcome  of  both  experiments  suggests  that  the  teams  effectively  applied 
these  procedures  during  testing  to  achieve  safe  ATC  operations  with 
reduced  aircraft  delays  emd  increased  operating  efficiency. 

During  debriefing,  the  participants  indicated  that  the  addition  of  Data  Link 
provided  an  opportunity  to  effectively  implement  a  “radar  team”  concept. 
However,  they  also  noted  that  standardized,  but  flexible  procedural 
guidelines  will  be  needed  in  the  future  to  ensure  that  two  and  three 
member  control  teams  will  be  optimally  productive  and  efficient  in  the  use 
of  Data  Link.  Such  guidelines  should  define  position  responsibilities  and 
intercoordination  requirements  for  team  members.  In  addition,  training  in 
team  work  may  be  needed  to  promote  effective  interaction  among 
individual  controllers  who  rotate  through  different  team  assignments. 

The  participants  also  argued  that  distributing  communications  tasks 
among  multiple  controllers  will  modify  the  information  requirements  of 
individuEd  control  positions.  As  noted  in  the  previous  section,  these 
requirements  will  include  the  addition  of  displays  for  associate  controllers 
to  evaluate  the  accuracy  of  their  inputs  and  monitor  transactions,  and 
provision  of  a  history  list  function  to  promote  team  coordination  and 
support  short  term  memory  limitations.  Finally,  while  not  technically 
feasible  in  the  current  en  route  workstation,  the  participants  noted  that 
enhancing  the  controller  interface  by  providing  the  radar  position  with  a 
third  keyboard/trackball  would  permit  all  members  of  a  three-controller 
team  to  make  Data  Link  entries. 

6.  BENEFIT  ESTIMATION  AND  AGGREGATION. 

The  primary  objective  of  this  study  was  to  identify  and  measure  some  of 
the  benefits  that  the  implementation  of  two-way  Data  Link  ATC 
communications  would  provide  to  NAS  users.  The  results  of  the  two 
experiments  presented  in  section  5  of  this  report  objectively  demonstrated 
that  controllers  using  two-way  Data  Link  were  able  to  provide  ATC 
services  that  improved  en  route  sector  productivity  and  efficiency.  These 
effects  were  reflected  in  reduced  aircraft  ground  delay,  flight  time,  and 
flight  distance  in  comparison  to  a  current  operational  environment  using 
only  voice  radio  communications. 

Estimating  the  economic  cost  savings  associated  with  these  findings 
provides  one  approach  to  determining  the  potential  magnitude  of  two-way 
Data  Link’s  impact  and  its  significance  to  NAS  users.  The  following 
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subsections  present  annualized  user  cost  savings  estimates  for  the  local 
Atlanta  ARTCC  sectors  tested  in  the  experiments,  and  for  a  group  of  en 
route  sectors  across  the  U.S.  where  operations  are  adversely  affected  by 
similar  limitations  in  ATC  communications  capabilities. 

6. 1  LOCAL  ATLANTA  ARTCC  BENEFIT  ESTIMATE. 

Measures  of  the  time  savings  achieved  by  aircraft  in  both  experiments  of 
this  study  were  used  as  the  basis  for  calculating  associated  cost  savings  in 
the  Atlanta  ARTCC.  Time  savings  were  expressed  in  decimal  minutes  for 
each  category  of  air  traffic  represented  on  the  historical  baseline  day. 

Table  8  summarizes  the  ground  and  airborne  time  in  sector  savings 
obtained  during  testing  with  Data  Link.  The  ground  delay  savings 
presented  in  the  table  are  the  total  minutes  for  all  affected  aircraft  at  the 
Atlanta  airport  achieved  by  eliminating  restrictions  on  aircraft  entering 
sector  32.  In-flight  savings  for  the  two  sectors  are  averages  for  the  test 
control  teams  in  the  three  replications  of  each  experiment. 

Estimates  of  the  user  costs  associated  with  these  results  were  computed 
according  to  Office  of  Management  and  Budget  (OMB)-approved  practices. 

TABLE  8.  TOTAL  GROUND  DELAY  AND  SECTOR  FLIGHT 
TIME  SAVINGS  FROM  EXPERIMENTS  1  AND  2 


Air  Traffic 
Category 

Number  of 
Aircraft 

Total  Minutes 
Saved 

Atlanta  Ground 
Delayed 

48 

1108.00 

Sector  32  Atlanta 
Departures 

24 

49.44 

Sector  32  Charlotte 
Departures 

4 

5.16 

Sector  32 
Overflights 

13 

14.69 

Sector  09 
Atlanta  Arrivals 

23 

95.45 

Sector  09  Peachtree 
Dekalb  Arrivals 

3 

.15 

Sector  09 
Overflights 

14 

29.26 
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Per-minute  operating  costs  were  based  on  data  obtained  from  the  most 
recent  update  of  the  FAA  Guide  for  Economic  Analysis  of  Investment  and 
Regulatory  Decisions  (FAA,  1994),  This  document  lists  average  aircraft 
operating  costs  weighted  by  the  hours  of  use  over  a  range  of  sizes  and 
types  of  aircraft.  For  commercial  air  carriers,  the  taxi  (ground  delay)  rate 
is  $26.20  per  minute.  The  airborne  rate  is  $31.50  per  minute. 

These  rates  for  the  “air  carrier  fleet”  represent  averages  for  equipment 
ranging  from  4-engine,  wide  body,  turbofan  aircraft  to  two-engine,  piston 
aircraft.  All  aircraft  that  were  flown  through  the  Atlanta  ARTCC  sectors 
during  the  test  periods  of  the  two  experiments  in  this  study  are  included  in 
this  category.  The  rates  are  conservatively  based  on  average  block-to- 
block  costs  including  the  value  of  fuel,  maintenance,  rent,  crew  salaries, 
and  other  direct  costs  to  air  carriers.  They  do  not  include  estimates  of  the 
value  of  passenger  time  loss  resulting  from  delays. 

Based  on  these  rates,  the  1,108  minutes  of  ground  delay  savings 
demonstrated  in  experiment  1  equates  to  a  dollar  cost  savings  with  Data 
Link  communications  of  $29,029.  The  total  of  194.15  minutes  of  in-flight 
time  savings  produced  in  both  experiments  yields  a  cost  savings  of  $6116. 
In  both  cases,  these  savings  refer  only  to  the  two  sectors  over  the  tested 
peak  traffic  periods  on  a  single  day. 

Annual  savings  estimates  for  these  traffic  rush  periods  were  based  on 
reports  of  the  frequency  with  which  the  two  sectors  experience  the  tested 
saturation  problems.  In  both  cases,  Atlanta  ARTCC  personnel  indicated 
that  these  problems  occurred  on  a  daily  basis  throughout  the  year.  In 
order  to  produce  a  conservative  estimate  of  annual  costs,  all  Saturdays  and 
five  major  holidays  were  disregarded  in  the  analysis.  The  remaining  period 
of  308  days  was  discounted  by  an  additional  20  percent  to  account  for  daily 
variations  in  traffic  demand,  leaving  a  final  total  of  246  problem  days  for 
the  two  rush  periods  in  sectors  32  and  09. 

Predictions  of  annual  cost  savings  were  based  on  the  product  of  the  daily 
savings  estimates  and  the  number  of  days  of  problem  occurrence. 
Accordingly,  the  cost  savings  attributable  to  reduced  ground  delays  for  the 
tested  Atlanta  sector  were  estimated  at  $7,141,282.  The  predicted 
combined  sector  savings  for  reduced  flight  time  were  $1,504,468.  Thus,  the 
total  estimated  annual  cost  savings  are  $8,645,750  for  the  two  sectors  in 
the  Atlanta  ARTCC. 
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It  should  be  noted  that  both  sector  32  and  sector  09  experience  rush 
periods  at  other  times  of  day  not  tested  in  the  present  experiments.  Thus, 
these  values  represent  a  conservative  estimate  of  the  total  annual  savings 
that  would  be  achieved  with  Data  Link  at  Atlanta. 

6,2  NATIONAL  BENEFIT  ESTIMATE. 

6.2.1  National  Sector  Survey. 

A  national  sector  survey  was  conducted  as  a  basis  for  producing  an  estimate 
of  the  annualized  national  economic  gains  that  would  be  associated  with  the 
results  of  this  study  (FAA  Technical  Center,  1994).  The  purpose  of  the  survey 
was  to  identify  and  gather  data  on  en  route  airspace  sectors  throughout  the 
NAS  which  experience  inefficiencies  attributable  to  communications 
problems  similar  to  those  found  in  the  Atlanta  ARTCC  sectors  tested  in  the 
study.  Specifically,  the  target  sectors  for  the  survey  were  those  which 
experience  volume  saturation  associated  with  frequency  congestion  and 
require  assistance  (e.g,,  MIT  restrictions,  aircraft  reroutes,  and  additional 
staffing). 

Telephone  interviews  were  conducted  with  Traffic  Management  Unit  (TMU) 
personnel  from  20  ARTCCs,  The  three  en  route  facilities  not  included  in  the 
survey  were  Anchorage,  Honolulu,  and  San  Juan,  After  explaining  the 
purpose  of  the  survey,  the  TMU  managers  were  asked  whether  their  facility 
contained  any  sectors  which  fit  the  target  sector  definition.  If  one  or  more 
sectors  were  identified,  the  interviewer  asked  the  manager  to  answer  the 
following  series  of  questions  about  each  sector: 

Sector  Identification  -  What  is  the  sector  number,  sector  identifier, 
sector  altitude  limits,  and  area  within  the  ARTCC  where  the  sector  is  located? 

Traffic  Types  -  What  categories  of  air  traffic  transition  through  the 
sector? 

Sector  Workload  and  Reroutes  -  How  busy  is  the  sector  and  what  level 
of  saturation  does  it  experience?  What  control  techniques  are  employed  to 
transition  the  aircraft?  Are  trackers  or  coordinators  used  regularly?  If  so, 
how  many  times  each  day?  Do  controllers  ever  hold  aircraft  because  of  sector 
saturation?  Does  TMU  ever  reroute  aircraft  around  the  sector  to  reduce 
saturation? 
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TMU  Restrictions  -  If  MIT  restrictions  are  implemented  to  reduce 
saturation  or  frequency  congestion  in  the  sector,  what  facilities  are  being 
affected  by  the  flow  and  what  is  the  specific  flow  restriction? 

Forty-one  additional  target  sectors  were  identified  whose  problem  profiles 
closely  resembled  those  of  the  two  Atlanta  ARTCC  test  sectors.  A  second 
telephone  survey  of  the  TMU  personnel  responsible  for  the  target  sectors  was 
completed  to  obtain  additional  quantitative  data  for  use  in  the  methodology 
developed  for  computing  national  benefits.  For  each  sector,  the  interviewee 
was  asked  to  (1)  identify  the  number  of  days  per  year  that  the  problem  is 
experienced,  (2)  specify  the  daily  time  period  during  which  the  problem 
occurs,  and  (3)  express  the  nature  of  the  air  traffic  situation  during  the  peak 
rush  period  by  estimating  the  extent  to  which  the  sector’s  problems  are 
associated  with  departures,  arrivals,  and  overflights. 

6.2.2  National  Benefit  Estimation  Methodology. 

Because  intensive  laboratory  testing  of  each  of  the  survey  target  sectors 
was  not  economically  feasible,  an  analytical  approach  was  used  to  extend 
the  results  of  the  present  study  to  a  national  level  by  extrapolation, 
comparison,  and  statistical  analysis.  Detailed  equations  used  in  the 
computation  of  economic  benefits  are  presented  in  appendix  B.  The 
following  paragraphs  summarize  the  methodology  in  narrative  form. 

The  process  of  estimating  benefits  began  with  an  analysis  of  the 
comparability  of  the  sector  saturation  problems  experienced  in  the  Atlanta 
ARTCC  test  sectors  to  the  41  additional  target  sectors  identified  by  the 
national  sector  survey.  The  two  test  sectors  in  which  efficiency 
improvements  were  observed  with  the  introduction  of  Data  Link  can  be 
classified  as  transitional  sections  of  airspace  in  which  groups  of  aircraft  are 
moving  between  flight  phases.  The  communications  and  control  problems 
in  the  sectors  are  associated  with  three  archetypal  configurations  of  air 
traffic  flow.  In  Atlanta’s  sector  32,  communication  limitations  adversely 
affect  the  ability  to  effectively  control  masses  of  aircraft  departing  from  an 
airport  that  must  be  climbed  and  separated  on  common  and  diverging 
routes.  In  sector  09,  several  streams  of  arrival  traffic  must  be  sequenced 
and  merged  over  an  arrival  fix.  The  communications  frequency  congestion 
problems  in  both  sectors  are  complicated  by  crossing  overflight  traffic. 

The  similarity  of  the  problems  experienced  by  the  target  survey  sectors  to 
those  of  the  test  sectors  was  quantified  by  comparing  their  traffic 
configurations  to  these  archetypes.  As  noted  in  section  6.2.1,  TMU 
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personnel  from  the  survey  ARTCCs  provided  expert  judgments  of  sector 
traffic  configuration  by  estimating  the  extent  to  which  the  air  traffic  in 
each  rush  period  could  be  classified  as  a  departure  problem,  an  arrival 
problem  and  an  overflight  problem.  Each  sector  was  characterized  by  its 
comparability  profile  expressed  as  percent  arrival,  departure,  and 
overflight  (e.g.,  80  percent  Arrival,  15  percent  Departure,  5  percent 
Overflight). 

In  performing  the  benefit  calculation,  these  profiles  were  used  to 
determine  the  magnitude  of  the  airborne  delay  savings  obtained  in  the 
experiments  that  would  be  attributed  to  each  sector.  Average  airborne 
delay  savings  for  the  test  sectors  were  calculated  for  each  category  of  air 
traffic  that  flew  in  the  two  Atlanta  scenarios.  A  per-aircraft  savings  for 
arrivals  of  4.15  minutes  was  based  on  the  reduction  in  delay  obtained  for 
Atlanta  arrivals  with  the  baseline  traiffic  in  experiment  2,  averaged  over  the 
three  test  controller  teams.  The  per  aircraft  savings  for  departures  of  2.06 
minutes  was  based  on  the  average  reduction  in  airborne  delay  for  the 
baseline  Atlanta  departures  in  experiment  1.  The  savings  for  overflight 
aircraft  of  1.63  minutes  was  based  on  a  weighted  average  reduction  in 
airborne  delay  for  the  overflight  aircraft  in  baseline  test  runs  from  both 
experiments. 

The  per-aircraft  airborne  delay  savings  for  each  survey  sector  problem 
period  was  computed  by  multiplying  the  sector’s  comparability  profile  by 
the  associated  delay  savings,  and  summing  the  products.  Using  the 
example  above,  the  sector  with  a  80  percent  arrival,  15  percent  departure 
and  5  percent  overflight  profile  would  yield  a  savings  of  .8(4.15)  -f 
.15(2.06) -I- .05(1.63)  =  3.71  air  minutes  per  aircraft. 

Total  airborne  delay  savings  for  a  survey  sector  problem  period  were 
computed  by  multiplying  the  traffic  count  by  the  per-aircraft  savings. 

The  FAA’s  Enhanced  Traffic  Management  System  (ETMS)  was  used  to 
obtain  exact  counts  of  the  number  of  affected  aircraft  in  the  survey  sectors 
during  each  problem  period.  Treiffic  count  data  in  consecutive  15-minute 
periods  for  a  single  day  were  obtained  from  the  ETMS  for  each  target 
sector.  The  data  were  searched  to  determine  sector  aircraft  counts  for  the 
problem  periods  identified  by  the  TMUs  in  the  sector  survey.  In  the 
example,  with  a  traffic  count  of  30  aircraft,  the  total  in-flight  savings  for 
the  sector  would  be  111.3  minutes. 

Savings  attributable  to  reduced  ground  delays  were  based  on  the  average, 
per-aircraft  time  savings  calculated  for  each  MIT  restriction  tested  in 
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experiment  1.  Assuming  an  ability  to  eliminate  restrictions  with  Data  Link, 
the  applied  savings  for  a  sector  enforcing  a  20  MIT,  15  MIT  or  10  MIT 
restriction  were  23.08,  12.52,  and  2.56  minutes,  respectively.  These  savings 
were  extended  only  to  those  sectors  for  which  the  national  sector  survey 
respondees  reported  that  MIT  restrictions  on  departures  for  entering  the 
sector  were  regularly  enforced  during  the  rush  period.  If  a  sector  was 
reported  as  applying  a  range  of  restrictions  over  the  period  (e.g.,  10-15 
MIT),  the  average  aircraft  savings  associated  with  the  lower  restriction  was 
applied.  If  the  sector  has  floating  restrictions,  no  ground  delay  benefit  was 
applied. 

Ground  delay  savings  for  each  problem  period  were  computed  by 
multiplying  the  per-aircraft  savings  from  the  experiment  by  the  problem 
profile  percentage  for  departures,  and  by  the  total  number  of  aircraft  in  the 
sector.  Thus,  assuming  a  20  MIT  restriction  in  the  example  sector,  this 
formula  yields  a  daily  savings  of  .15(23.08)30  =  103.86  minutes  saved  in 
ground  delays.  It  should  be  noted  that  the  estimates  produced  in  this 
manner  were  conservative  because  they  did  not  include  the  ground  delay 
savings  to  aircraft  in  the  departure  queue  which  do  not  enter  a  survey 
sector. 

The  projected  annual  delay  savings  for  each  sector  were  computed  by 
multiplying  the  in-flight  and  ground  delay  daily  estimates  by  the  number 
of  days  per  year  that  the  survey  respondees  reported  that  the  problem 
occurs.  In  cases  where  respondees  indicated  that  the  problem  was  a  daily 
occurrence  throughout  the  year,  a  conservative  estimate  was  used  by 
disregarding  all  Saturdays  and  five  major  holidays.  Thus,  the  maximum 
possible  number  of  problem  days  was  308. 

Total  monetary  savings  for  each  sector  problem  period  were  calculated  by 
multiplying  the  annual  minutes  of  in-flight  and  ground  delay  savings  for 
each  sector  by  the  FAA  per-minute  dollar  cost  rates  cited  in  section  6.1. 

For  commercial  air  carriers,  the  current  taxi  (ground  delay)  rate  that  was 
applied  was  $26.20  per  minute.  The  airborne  rate  was  $31.50  per  minute. 

6.2.3  National  Results. 

Detailed  delay  and  cost  savings  results  for  the  individual  en  route  sectors 
included  in  the  national  benefit  estimate  are  presented  in  appendix  B.  In 
summary,  the  data  indicate  that  ATC  performance  inefficiencies  associated 
with  the  inherent  limitations  of  the  broadcast  voice  radio  communications 
system  affect  9,679  flights  each  day  in  the  43  target  sectors.  According  to  the 
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calculations  performed  for  this  study,  these  inefficiencies  are  responsible  for  a 
total  of  7,980  24-hour  days  of  aircraft  delay  each  year  (1 1,491,387  minutes) 
that  could  be  eliminated  with  the  implementation  of  two-way  Data  Link  ATC 
communications.  This  is  equivalent  to  eliminating  delays  on  each  of  the 
affected  flights  by  one-half  day  in  the  air,  and  one-third  day  on  the  ground 
each  year.  As  shown  in  appendix  B,  the  total  estimated  annual  cost  savings 
for  NAS  users  would  be  over  337  million  dollars. 

Examination  of  appendix  B  shows  that  these  benefits  would  be  distributed 
throughout  the  target  sectors  and  ARTCCs.  The  two  sectors  with  the  highest 
potential  costs  savings  are  located  in  the  Chicago  and  Indianapolis  ARTCCs. 
Both  of  these  sectors  process  approximately  500  aircraft  during  the  daily 
problem  periods  identified  in  the  national  sector  survey.  Together,  they 
account  for  1,583  aircraft-days  of  delay  each  year.  The  combined  estimated 
savings  with  Data  Link  implementation  for  flights  in  these  sectors  is  over  63 
million  dollars  annually. 

In  the  context  of  overall  NAS  operations,  the  43  target  sectors  represent 
approximately  7  percent  of  the  total  number  of  domestic  en  route 
airspace  sectors.  Discounting  any  flights  that  traverse  more  than  one 
congested  sector,  the  9,679  affected  flights  comprise  approximately  10  percent 
of  the  average  daily  number  of  flights  to  large,  medium,  and  smaU  hub 
airports  in  the  U.S.  (FAA,  1993). 

Table  9  contrasts  the  annual  cost  savings  projected  for  two-way  Data  Link  in 
the  43  en  route  sectors  to  an  independent  estimate  of  current  airline  operating 
losses.  The  table  presents  data  from  a  recent  Air  Transport  Association  (ATA) 
analysis  which  indicated  that  annual  airline  losses  caused  by  ATC  and 
airspace  inefficiencies  are  approximately  $3.5  billion  (Swierenga,  1994).  As 
shown  in  the  table,  the  outbound  taxi  delay  and  en  route  loss  categories  that 
were  addressed  in  this  study  make  up  a  majority  of  the  total  airline  losses  due 
to  ATC  inefficiencies  (74  percent). 

The  benefit  estimates  for  these  categories  indicate  that  Data  Link  would 
reduce  outbound  taxi  delay  dollar  losses  by  9.9  percent,  and  en  route  losses  by 
15.8  percent.  The  total  benefit  estimate  attributable  to  the  en  route  sectors 
where  Data  Link  would  be  expected  to  expand  the  capacity  of  the  ATC 
communications  channel  and  alleviate  related  inefficiencies  is  approximately 
10  percent  of  the  total  ATA  annual  loss  estimate. 
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TABLE  9.  COMPARISON  OF  ANNUAL  DATA  LINK  BENEFIT 

ESTIMATE  TO  ATA  ESTIMATES  OF  AIRLINE  LOSSES 
FROM  ATC  AND  AIRSPACE  INEFFICIENCIES 


Delay  Category 

ATA  Estimate  of 
Annual  Airline  Losses 

Estimated  En  Route 
Annual  Data  Link 
Savings 

Gate  Delays 

Flow  Control 

$  95,137,222 

Air  Traffic 

$  73,902,504 

Airport 

$  7,390,226 

Taxi  Delays 

Outbound 

$  1,215,547,622 

$  119,790,977 

Inbound 

$  361,034,841 

En  Route  Losses 

$  217,955,195 

Indirect  Routes 

$  1,282,705,520 

Execution  and  Delays 

$  97,308,764 

$  72,741,148 

Weather 

$  28,696,818 

TOTAL 

$  3,492,737,393 

$  337,746,172 

In  evaluating  the  reliability  of  the  benefit  projections  discussed  above,  it 
should  be  noted  that  extensive  measures  were  taken  to  minimize  the 
probability  of  overestimating  actual  cost  savings.  These  measures 
included  the  use  of  objective  data  from  an  operational  ATC  facility  and 
from  extremely  high  fidelity,  manned  simulation  research  as  the  basis  for 
calculating  savings.  In  addition,  conservative  options  were  consistently 
adopted  when  screening  sectors  for  inclusion  in  the  national  calculation, 
and  when  selecting  the  time  periods  and  number  of  flights  to  which 
benefits  were  extended. 

In  particular,  it  should  be  remembered  that  this  study  addressed  only  one 
dimension  of  Data  Link’s  potential  benefit.  The  estimated  cost  savings  are 
restricted  to  Data  Link’s  ability  to  increase  ATC  communications  capacity 
in  a  single  domain  of  ATC  operations.  Safety  and  efficiency  enhancements 
that  may  result  from  two-way  Data  Link’s  capability  to  improve  the 
accuracy  of  communications,  and  from  its  use  in  other  ATC  environments 
are  expected  to  significantly  increase  estimated  cost  savings  to  aircraft 
owners  and  operators. 
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7.  CONCLUSIONS. 


The  resijlts  of  the  study  presented  in  this  report  support  the  following 
conclusions  regarding  the  effects  of  domestic,  two-way  Data  Link  Air  Traffic 
Control  (ATQ  services  on  flight  delays,  flight  efficiency,  and  associated 
National  Airspace  System  (NAS)  user  costs. 

a.  The  real-time,  manned  simulation  experiments  performed  for  this 
study  indicate  that  the  implementation  of  two-way  Data  Link  ATC  services  in 
an  en  route  ATC  environment  can  significantly  reduce  flight  delays  and 
operating  inefficiencies.  The  study  supported  the  hypothesis  that 
supplementing  the  voice  radio  communications  system  with  Data  Link  in  two 
operational  airspace  sectors  in  the  Atlanta  Air  Route  Traffic  Control  Center 
(ARTCC)  alleviated  sector  saturation  problems  that  are  experienced  on  a 
regular  basis.  Specific  conclusions  drawn  from  the  experiments  are  outlined 
below. 


1 .  Data  Link  permitted  the  elimination  of  spacing  restrictions  that 
are  enforced  to  prevent  saturation  in  a  departure  sector.  This  resulted  in  a  62 
percent  reduction  in  groimd  delays  for  departing  aircraft. 

2.  The  implementation  of  Data  Link  significantly  enhanced  sector 
throughput  and  flight  efficiency.  In  both  the  departure  and  arrival  sectors, 
average  flight  times  and  distances  were  reduced  by  approximately  20  percent. 
These  improvements  were  associated  with  more  efficient  climb  profiles  for 
departures  and  the  elimination  of  aircraft  holding  patterns  and  inefficient 
vectoring  for  arrivals. 

3.  The  study  showed  that  controllers  using  Data  Link  also  were  able 
to  maintain  more  efficient  control  of  the  sector  traffic  when  the  number  of 
departure  and  arrival  aircraft  were  increased  by  10  to  40  percent  over  current 
tr^c  levels.  In  addition,  the  data  suggest  that  the  rate  of  growth  in  aircraft 
delays  with  increased  traffic  levels  representative  of  future  airspace  demands 
would  be  significantly  curtailed  with  Data  Link  in  comparison  to  a  voice-only 
communications  environment. 

4.  The  evidence  for  user  benefits  obtained  in  the  experiments  was 
validated  by  multiple  supplementary  criteria.  As  indicated  by  measures  of 
aircraft  separation,  and  operational  assessments  by  ATC  supervisors, 
controllers  and  pilots,  all  test  runs  were  completed  with  a  margin  of  safety 
which  equaled,  or  exceeded  typical  levels  observed  in  the  operational  sectors. 
Controller  performance  on  key  sector  tasks  was  unanimously  graded  by  ATC 
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supervisors  as  being  within  normal  limits,  or  better.  Furthermore,  controller 
ratings  and  measures  of  handoff  latency  yielded  no  evidence  of  excessive 
controller  workload  with  Data  Link. 

b.  The  current  operational  inefficiencies  of  both  sectors  examined  in 
this  study  are  caused  by  limitations  on  controller  abilities  to  communicate 
effectively  with  aircraft  in  a  congested  voice  radio  environment.  The  benefits 
that  were  recorded  in  the  experiments  were  a  direct  result  of  the  increased 
communications  capacity  made  possible  by  adding  Data  Link  to  the  existing 
voice  channel. 

The  findings  of  the  study  indicate  that  resulting  improvements  in  sector 
productivity  and  in  ATC  service  to  aircraft  were  attributable  to  three  primary 
factors.  First,  Data  Link  alleviated  frequency  congestion,  making  the  voice 
radio  consistently  available  for  time-critical  clearance  delivery.  Second,  a 
majority  of  standard  clearances  and  other  repetitive  messages  were  sent  using 
simplified  Data  Link  inputs,  thereby  freeing  the  controllers  to  devote  more 
time  to  developing  and  executing  effective  control  strategies.  Finally,  optimal 
use  of  the  expanded  communications  capability  was  achieved  by  distributing 
communications  tasks  to  all  members  of  the  control  teams.  This  permitted 
simultaneous  messaging  to  different  aircraft.  It  also  allowed  the 
controllers  to  act  as  coordinated  and  flexible  team  decision  makers. 

c.  Direct  computation  of  the  annual  dollar  value  associated  with 
the  performance  results  obtained  for  the  sectors  during  the  two,  peak 
traffic  periods  tested  in  the  study  yielded  evidence  for  significant  user  cost 
savings.  Total  annual  savings  attributable  to  the  reduced  ground  delays 
and  flight  times  demonstrated  with  Data  Link  exceeded  $8.6  million  for  the 
two  sectors  at  the  Atlanta  ARTCC. 

d.  It  is  very  probable  that  the  efficiency  problems  addressed  in  this 
study  and  the  observed  impact  of  Data  Link  extend  beyond  the  two  Atlanta 
ARTCC  sectors  that  were  tested.  A  national  sector  survey  conducted  in 
conjunction  with  the  study  shows  that  at  least  41  additional  en  route 
sectors  across  the  U.S.  experience  similar,  communications-related 
saturation  problems  during  one  or  more  daily  periods  of  peak  traffic.  The 
survey  indicates  that  over  9,600  flights  annually  experience  significant 
airborne  and/or  ground  delays  caused  by  communications  limitations  in 
these  sectors.  Benefit  estimates  which  extended  the  results  of  the  study  to 
the  survey  sectors  show  that  these  flights  would  achieve  a  total  of  over 
$337  million  in  operating  cost  savings  annually  'with  the  introduction  of 
two-way  Data  Link  ATC  communications. 
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8.  RECOMMENDATIONS. 


The  following  recommendations  for  future  actions  by  the  Federal 
Aviation  Administration  (FAA)  and  by  aircraft  owners  and  operators  are 
based  on  the  results  and  conclusions  derived  from  this  study. 

a.  This  study  produced  data  which  show  that  the  implementation  of 
domestic,  two-way  Data  Link  communications  will  result  in  significantly 
improved  Air  Traffic  Control  (ATQ  service  and  reduced  aircraft  operating 
expense.  The  extent  to  which  such  benefits  will  be  realized  in  the  National 
Airspace  System  (NAS)  will  depend  on  the  number  of  aircraft  that  are 
equipped  with  the  necessary  Data  Link  avionics  systems.  Consequently,  it  is 
recommended  that  aircraft  owners  and  operators  consider  the  findings  of  this 
study  as  they  perform  cost-benefit  analyses  in  support  of  any  decision  to 
invest  in  the  capability  to  receive  two-way  Data  Link  ATC  services. 

b.  Two-way  Data  link  is  expected  to  directly  impact  the  quality  of  ATC 
operations  by  increasing  the  capacity  of  the  controller-pilot  communications 
channel  and  by  improving  the  accuracy  of  air-ground  communications. 

The  benefits  and  estimated  cost  savings  demonstrated  in  this  study  are 
associated  only  with  the  ability  of  Data  Link  to  increase  communications 
channel  capacity  in  a  single  domain  of  ATC  operations.  Safety  and 
efficiency  enhancements  that  may  result  from  two-way  Data  Link’s 
capability  to  improve  the  accuracy  of  communications,  and  from  its  use  in 
other  ATC  environments  are  likely  to  significantly  increase  estimated  cost 
savings  to  cdrcraft  owners  emd  operators.  For  this  reason,  it  is 
recommended  that  the  FAA  conduct  research  to  obtain  evidence  for  these 
additional  benefits  and  to  estimate  the  magnitude  of  related  cost  savings. 
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APPENDIX  A 


ASSESSMENT  INSTRUMENTS 


DATA  LINK  BENEFITS  STUDY 
SUPERVISOR  QUESTIONNAIRE 


Part  I.  Performance  Ratings 

Please  evaluate  the  ATC  operations  observed  during  this  test  run  on  the  following  factors: 

1.  Errors  or  omissions  in  required  flight  strip  marking. 

a)  _  Never  Occurred 

b)  _  Rarely  Occurred 

c)  _  Occurred,  But  Within  Normal  Limits  of  Operational  Acceptability 

d)  _  Occurred  More  Often  Than  Normal  For  This  Sector  At  This  Time  of  Day 

e)  _  Occurred  Unacceptably  Often 

2.  Gave  Arriving  Aircraft  Inefficient  (Early)  Descent. 

a)  _  Never  Occurred 

b)  _  Rarely  Occurred 

c)  _  Occurred,  But  Within  Normal  Limits  of  Operational  Acceptability 

d)  _  Occurred  More  Often  Than  Normal  For  This  Sector  At  This  Time  of  Day 

e)  _  Occurred  Unacceptably  Often 

3.  Gave  Departing  Aircraft  Inefficient  (Late)  Climb. 

a)  _  Never  Occurred 

b)  _  Rarely  Occurred 

c)  _  Occurred,  But  Within  Normal  Limits  of  Operational  Acceptability 

d)  _  Occurred  More  Often  Than  Normal  For  This  Sector  At  This  Time  of  Day 

e)  _  Occurred  Unacceptably  Often. 
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4.  Issued  Clearances  Later  or  Earlier  Than  Appropriate. 


a)  _  Never  Occurred 

b)  _  Rarely  Occurred 

c)  _  Occurred,  But  Within  Normal  Limits  of  Operational  Acceptability 

d)  _  Occurred  More  Often  Than  Normal  For  This  Sector  At  This  Time  of  Day 

e)  _  Occurred  Unacceptably  Often 

5.  Failed  to  Comply  with  Letters  of  Agreement. 

a)  _  Never  Occurred 

b)  _  Rarely  Occurred 

c)  _  Occurred,  But  Within  Normal  Limits  of  Operational  Acceptability 

d)  _  Occurred  More  Often  Than  Normal  For  This  Sector  At  This  Time  of  Day 

e)  _  Occurred  Unacceptably  Often 

6.  Offered  Hand-Offs  Earlier  Than  Appropriate. 

a)  _  Never  Occurred 

b)  _  Rarely  Occurred 

c)  _  Occurred,  But  Within  Normal  Limits  of  Operational  Acceptability 

d)  _  Occurred  More  Often  Than  Normal  For  This  Sector  At  This  Time  of  Day 

e)  _  Occurred  Unacceptably  Often 

7.  Offered  Hand-Offs  Later  Than  Appropriate. 

a)  _  Never  Occurred 

b)  _  Rarely  Occurred 

c)  _  Occurred,  But  Within  Normal  Limits  of  Operational  Acceptability 

d)  _  Occurred  More  Often  Than  Normal  For  This  Sector  At  This  Time  of  Day 

e)  _  Occurred  Unacceptably  Often 
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8.  Accepted  Hand-Offs  Later  Than  Appropriate. 


a)  _  Never  Occurred 

b)  _  Rarely  Occurred 

c)  _  Occurred,  But  Within  Normal  Limits  of  Operational  Acceptability 

d)  _  Occurred  More  Often  Than  Normal  For  This  Sector  At  This  Time  of  Day 

e)  _  Occurred  Unacceptably  Often 

9.  Failed  To  Meet  Miles-In-Trail  Restrictions  That  Were  In  Force  During  The  Test  Run. 

a)  _  Never  Occurred 

b)  _  Rarely  Occurred 

c)  _  Occurred,  But  Within  Normal  Limits  of  Operational  Acceptability 

d)  _  Occurred  More  Often  Than  Normal  For  This  Sector  At  This  Time  of  Day 

e)  _  Occurred  Unacceptably  Often 

Part  II.  Overall  Operational  Assessment 

Please  use  the  scale  below  to  make  an  overall  operational  assessment  ofATC  performance 
during  this  test  run: 

a)  _  The  margin  of  safety  was  higher  than  normal  for  operations  at 

this  sector  at  the  Atlanta  ARTCC. 

b)  _  Operations  were  typical  for  this  sector,  with  acceptable  safety. 

c)  _  Operational  safety  was  not  compromised,  but  I  had  safety 

concerns. 

d)  _  Operations  were  unsafe,  and  unacceptable. 

If  you  checked  c.  or  d.,  please  explain  your  rating  below.  Thoroughly  describe  the  incidents  or 
factors  which  influenced  your  judgment. 
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Part  III.  Data  Link  Usage 

1.  Did  you  observe  any  input  errors  in  selecting  or  entering  Data  Link 
messages? 

a)  _  I  did  not  notice  any  input  errors. 

b)  _  I  noticed  a  few  errors. 

c)  _  The  controllers  made  several  errors. 


2.  If  you  observed  Data  Link  entry  errors,  how  were  they  handled? 

(Check  all  that  apply) 

a)  _  Errors  were  detected  by  the  controllers  during  the  input 

process  and  corrected  before  sending. 

b)  _  The  controllers  noticed  the  error  in  the  FDD  or  Status  List 

Display  after  sending  the  message.  The  error  was 
corrected  by  voice  radio  or  a  new  Data  Link  message. 

c)  _  The  error  was  detected  only  by  noticing  an  unintended 

aircraft  maneuver. 

d)  _  The  error  was  never  detected  by  the  controllers. 

3.  From  your  observations,  do  you  feel  that  Data  Link  communications 
will  present  a  significantly  greater  potential  for  undetected  controller 
error  than  voice  communications  in  an  operational  environment? 
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DATA  LINK  BENEFITS  STUDY 
CONTROLLER  QUESTIONNAIRE 


Part  I.  Position  Duties  and  Workload 

1.  Duty  Description  -  Please  indicate  the  extent  to  which  you  performed  each  of  the  following 
tasks  at  your  position  during  this  test  run.  For  each  duty,  place  an  “x”  in  the  column  that  best 
describes  your  level  of  task  involvement. 


Always 
My  Duty 


Mostly  Occasionally  Rarely 
My  Duty  My  Duty  My  Duty 


Never 
My  Duty 


Issued  ATC  Clearances 
Using  Voice  Radio 

Issued  ATC  Clearances 
Using  Data  Link 

Offered/Accepted  Sector 
Handoffs 

Sent  TOC  Messages 
Using  Voice  Radio 

Sent  TOC  Messages 
Using  Data  Link 

Marked  Flight 
Strips 

Monitored  PVD  for 
Aircraft  Conformance 

Monitored  PVD  for 
Aircraft  Conflicts 

Monitored  Data  Link 
Transactions 

Handled  Land  Line 
Communications 

Directed  Others  on 
Control  Team 


Made  Control 
Decisions 
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Coordinated  My  Actions 
With  Other  Team  Members 


Performed  Sector 

“Housekeeping”  _  _  _  _  _ 

Other  Duties 

(Describe  Below)  _  _  _  _  _ 

2.  Workload  Rating  —  Please  use  the  scale  below  to  rate  the  level  of  workload  that  you 
experienced  at  your  position  during  this  test  run.  This  should  be  a  rating  of  your  personal 
perception  of  how  hard  you  had  to  work  to  perform  your  duties  -  not  an  estimate  of  overall 
sector  loading  or  traffic  count.  Place  an  “x”  in  the  space  next  to  the  phrase  which  best 
describes  your  perceived  workload. 

In  comparison  to  a  corresponding  work  period  at  this  sector  in  the  facility,  the  workload  at 

my  position  during  this  test  run  was: 

_  Much  Lower  Than  Usual 

Somewhat  Lower  Than  Usual 


_  About  The  Same 

_  Somewhat  Higher  Than  Usual 

_  Much  Higher  Than  Usual 

Please  describe  any  factors  that  you  feel  may  have  influenced  your  perceived  workload: 

Part  II.  Overall  Operational  Assessment 

Please  use  the  scale  below  to  make  an  overall  operational  assessment  of  ATC  performance  at 
your  sector  during  this  test  run: 

a)  _  The  margin  of  safety  was  higher  than  normal  for  operations  at 

this  sector  at  the  Atlanta  ARTCC. 

b)  _  Operations  were  typical  for  this  sector,  with  acceptable  safety. 

c)  _  Operational  safety  was  not  compromised,  but  I  had  safety 

concerns. 

d)  _  Operations  were  unsafe,  and  unacceptable. 
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DATA  LINK  BENEFITS  STUDY 
PILOT  SAFETY  QUESTIONNAIRE 

What  is  your  opinion  about  the  safety  of  the  flight  you  just  completed? 

1  2  3  4  5  6  7 

Completely  Completely 

Safe  Unsafe 

Please  explain  any  factors  that  affected  the  safety  of  this  flight: 


A-7 


APPENDIX  B 


NATIONAL  BENEFIT  CALCULATIONS 


NATIONAL  ESTIMATION  EQUATIONS 


Subscripts  indicate  the  identity  of  the  reference  case  sector  or  of  the 
configuration. 

Step  one:  computing  benefits  from  baseline  sectors 
1a.  Ba9=  Arrival  air  minutes  saved 
Arrival  aircrafts 


C  BdePT  =  Bd32 

Step  three:  determine  the  make-up  of  a  congested  national  sector. 

3.  Config(Sector  X)  =  (%  Arrival  problemx) 

+  (%  Departure  problemx) 

+  (%  Cross  problemx) 

Step  four:  apply  empirical  aircraft  delay  savings  according  to  the  proportions  in  the 
congested  national  sector. 

4a.  Config.  Savings(Sector  X)  =  (%  Arrival  problemx)  *  Barr 

+  (%  Departure  problemx)  *  (Bdept-air  +  0"mit  *  Bqept-gr) 
+  (%  Cross  problemx)  *  Bcross 


Note  0"mit  is  a  0/1  variable  indicating  positive  ground  delays  when  a  flow 
restriction  imposes  delays  on  a  major  airport. 


Scale  the  per  aircraft  delay  by  the  number  of  aircraft  in  the  congested  national  sector 
during  the  defined  period  of  congestion 
4b.  Benefitx  =  Config.  Savingsx  *  aircraftx 

=  %  Arrival  problemx*  aircraftx  *  Barr 

+  %Departure  problemx*  aircraftx  *  (Bdept-air  +  0"mit  *  Bdept-gr) 

+  %  Cross  problemx*  aircraftx*  Bcross 


Step  five:  find  the  annual  burden  of  the  problem 

5.  Annual  Benefitx  =  Benefitx  *  Frequency  that  congested  period  occurs  in  a  yearx 
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step  six:  apply  the  economic  values  of  minutes  of  delay 
6.  $  of  Annual  Benefit  =  Days/year  * 

[%  Arrival  problem/*  aircraftx  *  Barr  *  Vair 
+  %Departure  problem/*  aircraftx  *  (Bdept-air  *  Vair  +  O^mit  *  Bdept-gr  *  Vground) 
+  %  Cross  problem/*  aircraftx  *  Bcross  *  Vair  ] 
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NATIONAL  BENEFIT  RESULTS 
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